Will Net Neutrality Kill MPLS?

Net Neutrality proposes that all Internet packets are treated equally - no buying a so-called “fast lane” where delivery is virtually guaranteed. Broadcasters and other real-time data users depend on the MPLS protocol and pay extra for packet reliability - but is that really over the “Public Internet.” We’re talking about it with Bob Holowenko on TWiRT.

Watch the Video!

Read the Transcript!

Announcer: This Week in Radio Tech, Episode 242 is brought to you by Axia Audio and the new Fusion AoIP mixing console. Active features and capabilities refined from over a decade's worth of IP audio experience. By the Telos ProSTREAM X/2 and 9X/2 audio processing and streaming coding software with adaptive streaming technology. And by Lawo and the new crystalCLEAR virtual radio console. CrystalCLEAR is the radio console with a multi-touch touchscreen interface.

A net neutrality proposes that all internet packets are treated equally, no buying a so-called fast lane where delivery is virtually guaranteed. Broadcasters and other real-time data users depend on the MPLS protocol and pay extra for packet reliability, but is that really over the public internet? We're talking about it with Bob Holowenko on TWiRT.

Kirk Harnack: Hey, welcome in to This Week in Radio Tech. I'm Kirk Harnack, your host, delighted that you're here. This is show #242 of This Week in Radio Tech. Been going strong for a long time.

And as usual, I have a co-host with me on the show today. It's Chris Tobin. And at least for part of the show - don't know if he'll be able to stick around for the whole show - we've got Bob Holowenko, who is one of the sharpest IT guys I think you'll ever meet. And you know what? This guy knows IT and RF, and something about audio, too. So he's always great to talk to.

Let's bring in right now, though, the best-dressed engineering in radio from his office in Manhattan. Hello there, Chris Tobin. Welcome in, buddy.

Chris Tobin: Hello, Kirk. Yes, it's a Manhattan office. It's a home office. Soho, I guess, right? So it should be good. Nice and warm outside. About 20...

Kirk: Is it really?

Chris: No. It's only 30 degrees. It's cold out.

Kirk: Well, it's not like that arctic bomb we had, what, a week, week and a half ago.

Chris: Oh, yes. The other night, I think it was actually single digits here in Manhattan. It was quite cold.

Kirk: I see you're close to the heater, though.

Chris: Yes. The radiator definitely will come on if necessary. But actually, I've been very fortunate. We've had some new windows installed and the drafts are no longer there, so we've actually increased the room temperature by about 10 degrees.

Kirk: Oh, wow.

Chris: So it's nice. Yes, we haven't had to use the heat all winter except for two nights ago when it was down to 9 degrees. It was nice.

Kirk: And see, I'm just not classy enough to wear shirts that have polo horses on them. How do you get away with that?

Chris: Well, it was either that, or... I was working outdoors today at an AM transmitter site, so that outfit would have been really grungy looking. But then again, maybe it would be suitable for this.

Kirk: That's right. Well, then I couldn't call you best-dressed, could I?

Chris: Well, it depends. Yeah, I guess, yeah, it would be a problem.

Kirk: Hey, this is the show where we talk about radio engineering. Everything technical, from the microphone to the light bulb at the top of the tower. Our show is brought to you by the folks at Lawo and the Lawo crystalCLEAR virtual radio console. Also brought to you by a very cool new division of the Telos Alliance called ProSTREAM. We'll be telling you a little bit about ProSTREAM and a couple new products that they've come out with. And also brought to you by Axia, the Axia Fusion audio console.

I've got something that you've never seen before coming up at the end of the show. It's our friend Clark Novak. Those of you on the west coast probably know Clark as a radio programmer, music director, and Clark is going to be introducing you to some really cool features of the Fusion console. So stick around for that near the end of our show, because that's when we're going to have Clark on for a few minutes.

All right. This this show is, like I said, a bit of a potpourri. But I think the subject we're going to start with, and maybe stick with, is the subject of net neutrality and how it might affect broadcasters.

I was kind of searching, casting about the interwebs today, and that's always, I guess, a dangerous thing, and came upon a blog post. You know, everybody's an expert. Everybody's a pundit, right? So we don't know how accurate this might be. And the speculation may be just that speculation. But it is a blog post from the Networking Nerd. He's, I assume it's he, I'm sorry, I don't know. It may be a female. But NetworkingNerd.net is the website.

This blog post is from November of last year, so just a few months ago. "The Trap of Net Neutrality." And sounds like he's going to be negative on net neutrality, but turns out he's positive on it. But he's got a comic in this. It says, "When all the packets are treated as EF, none of them are."

And I want to bring in somebody who can explain what that means, Bob Holowenko from Canada. Bob, where are you in Canada?

Bob Holowenko: Basically central British Columbia. I'm in an area known as the Caribou Region. So not a lot of people are going to know where that is. But if they know anywhere near, I don't know, Prince George, which is kind of the center of BC, about 100 and so kilometers from there.

Kirk: Oh, wow. Okay. Sounds like the middle of nowhere. But you have internet, obviously.

Bob: I've got a couple different types of internet, all directly to my place, and I switch between them as needed.

Kirk: Bob, you've been on the show before. And you also, just to let folks know, Bob knows a lot about RF. He's helped me play with one of these little RF analyzers, the kind, the little USB stick that plugs into your PC, and you run some software and analyze, you know, what's in the air around you. And I have, I'm in a basement right here, so I haven't got as much RF in here as I'd like to get. But Bob, thank you for that.

I just want to let folks know that, you know, you kind of are an expert at this. You work at an ISP and you do broadcast engineering. You know a lot about internet telephony. So I think you're pretty well qualified to at least speculate as well as anybody else does on these issues.

So I just read this thing. "When all the packets are treated as EF, none of them are." What does that mean?

Bob: So it's a reference to QoS, essentially. There's different layers of QoS, or different tiers of QoS, and EF stands for Expedited Forwarding, which is just, it's marketing speak, essentially for the best class, the best case of traffic. And EF is a tag that's used for VoIP.

What a lot of people don't, well, what a lot of people hear is that packets such as VoIP, or anything that's super important - although it's supposed to be left alone for VoIP - that's tagged with the tag EF using a thing called differentiated services code point. So it gets a special marker that says 46. Okay, you're EF. That traffic gets precedence through a lot of mid-range gear that's not necessarily QoS-aware, but it'll work well enough to recognize that that's a certain type of traffic.

It's used a lot in enterprise networks, carrier networks, to give precedence to voice calls so that you don't end up in a situa... where... it... and... ooh... because, you know, that wouldn't be easy to listen to. It wouldn't be fun to listen to.

I do that on the phone sometimes, and people actually think it's...

Bob: Right. You scared me.

Kirk: Skype's fine. But now, you mentioned, I want to back up a hair. Of course, I work for the folks at Telos, and so I'm familiar with Telos's implementations, but that doesn't mean I have a wide view of the whole world and what the standards are out there, which is why we bring expert guests on, and then we have an expert co-host, Chris Tobin, who sees a lot of different stuff. I've got blinders on.

One of the pieces of gear from Telos, called the iPort, does have a setting for DSCP class of service. And you just mentioned this polysyllabic phrase differentiated service - differentiated service code point. Can you... that sounds like a lot of words that just means how important I am.

Bob: Well, yeah. It basically means how important the specific traffic is. And it doesn't necessarily mean a lot until you're using it. DSCP is just a, it's a standard set of recognized numbers. Different hardware's going to do it in different ways. But, you know, in the end, it all means the exact same thing. It's part of a standard. Like, it's 802.1p, or something along those lines, and it runs at the IP layer.

And it's basically just a tag that is used for, if you've got a saturated link, or a link that is policed down, where you're actually shaping it to a specific speed. So say at one of your offices, you've got a 100 MB pipe. At another office, it's only a 30 MB pipe. Your service would be policed down to 30 MBs by your carrier.

To be able to say that I want this traffic to be able to pass freely without having somebody's YouTube upload or download impact it, then you would mark it as such. So these little tags are basically there as a, well, it's like the Titanic. Everybody had their own class as to what layer of the ship they got.

Kirk: Aha.

Bob: In this case, it's the exact same thing. You don't want to be in the one that sinks first, or gets packet dropped first, tail dropped.

Kirk: So, by the way, we had a little bit of audio trouble there, but I think it may have corrected now. So, okay, it sounded like that USB problem with the audio gear sometimes.

So hey, I was just looking up on Wikipedia about differentiated services, and I see the phrase "diff serv" mentioned.

Bob: Mm-hmm.

Kirk: I've heard that before. Does that mean anything, the same as DSCP? Is that, tell me how those two terms are related.

Bob: Yeah. It's basically diff serv, differentiated services. So they basically mean the exact same thing.

Kirk: Okay. All right.

Bob: They do mean the same thing. It's just slang for the other.

The point, though, that you mentioned the comic to begin with, where it says, "When all of the packets are marked as EF, none of them are." It basically means that, you know, if you're overdoing it, if you're throwing everything out as high priority, then you're not really doing anything.

A lot of people get worried when they see packet drops of lower precedence traffic, because they think, well, you know, there's something wrong with the line. There's something wrong with the service. But technically, your QoS isn't working if it's not dropping packets. Of course, you know, you might not be saturated at that time.

But if you're highly saturated your link and you're not seeing certain types of traffic get more drops than another type of traffic, depending on how you set it up - I realize there's a lot of asterisks that apply around that. But technically your QoS isn't doing anything for you.

Kirk: So I want to ask Chris Tobin to step in for a second. So Chris, you know, my experience is pretty much only with the brand of gear that the company I work for. You, however, do consulting with all kinds of brands of gear. What's your view of implementation of DSCP specification in broadcast gear?

This is where we're going to go with all this. It's, you know, broadcasters, just like everybody else, want their message to get through the network first without interruption, without being interrupted by somebody else's email or BitCoin transaction.

So what's your experience in all this, Chris?

Chris: Well, I would say that the manufacturers, the hardware side of things, they've got it working. It works just fine when you do implement it. I think Bob's point is best taken is how you design your network, the architecture. You know, if you saturate the network, and you don't have any dropped packets, and your QoS has been implemented, there's something going on.

I can say from personal experience working on several projects where we did VLANs throughout the facility to offer up with diff serv and everything else going on, ways of trying to get those BitCoin, email traffic things off the, out of the way and this stuff worked fine.

I think really what needs to be done for broadcast is looking at this... don't look at the EF, don't look at the diff serv, don't look at that just by itself. Look at the bigger picture, which is your network. You know, the highways, the off ramps, the on ramps, and make sure you're managing all that. As Bob pointed out, the asterisks that are part of all this.

Because I could take, say, a Telos box and implement QoS on that, and take a Comrex box, implement QoS on that, then take a Cisco box and implement QoS on that. But if the networks in between those boxes are totally upside down, nothing's going to happen right.

So it's more than just one or two pieces, now, of the puzzle. It's the entire puzzle you have to really understand and work with.

I mean, I could be wrong, but that's what I've experienced, and that's how I've found best to get around some of the issues. Like you say, if somebody's downloading a video or taking email, and all of a sudden my IP packets are getting screwed up for audio and getting dropouts, you know? That shouldn't happen if you do it right.

But maybe I'm wrong. Bob will tell me otherwise. But I'm pretty sure you can definitely manage it properly.

Bob: Your network has to be at least in a stable state before you end up trying to layer additional services like...

Chris: Exactly.

Bob: ...like Voice over IP on top of them. Some of the problems that networks have where you've got quite a bit of traffic or quite a bit of jitter because of saturation, or because of really bursty protocols, QoS can sort of help with that.

But to your point, it's often lipstick on a pig. It doesn't always do... It's not the magic wand. QoS can do a lot of amazing things, and when it's used right, it can certainly be fine-tuned enough to make almost anything work. But if you've got a network that's got a half-duplex link in the middle, or, I don't know, some piece of technology that can't keep up, QoS isn't necessarily going to do much for you.

The packets need to be able to get to the next hop before they can even be analyzed as a packet that's tagged. So you can't take something that's in very rough condition and make it nicer. You can take something and keep it from getting worse, but that's about it.

Kirk: Hey, Bob, you said something that I want you to expand on after we take a quick commercial break. That is you said the network itself has to be stable before you add other things to it. I'm absolutely convinced that whether we're talking about a local audio over IP system, or if we're talking about, over distance, IP audio or IP video communications, that we've experienced this. Customers of Telos and other brands have experienced this, too. They hook up two things and, by golly, they don't work very well. Or they hook up their VoIP phone, and it's... oh... I....are... instead of, you know, nice smooth communications.

And so just like you said, you can't just put lipstick on the pig by, "Okay, I've got a crummy network here, things are going badly, but if I prioritize these packets, then it's going to work better, at least for these things." That probably is not the case.

So when we come back from the commercial, I want to hear from you, if you can tell us what are some of the things that we, as engineers, can learn to do. What are some tools we can deal with, what are some ways that we can tell whether our LAN and our connection to the outside world is saturated, is working as well as it could. I'm not sure I know how to best tell that.

So we're going to come right back and hear your thoughts about that.

Our show is brought to you in part by the folks at Lawo and the crystalCLEAR virtual radio mixing console. I'm excited about this console. It's pretty cool. I've got to tell, I mean, I have a personal reason why. Twenty years ago - longer than that, I think - I was talking to a console manufacturer. "Man, what if we could put a console on a touchscreen?"

Now, 20 years ago, touchscreen technology was in its infancy, and it just wasn't practical at all to do that. But somebody's gone and done this, and it's the folks at Lawo. You know, they make really big consoles for remote trucks and TV studios and live set, big live sound installed venues. Well, they also make a couple of small consoles for radio, and the crystalCLEAR is one of them.

It's a virtual radio mixing console, in the sense that the surface that you, the talent, is working with, all exists on a touchscreen. It's a multi-touch touchscreen. So that means you can look at it, you can touch two or three faders at the same time while you hit a button on or off. You can actually make ten touches all at the same time, since that's how many fingers most of us have, that's about the right number. You know, me, I can probably only manage two things at once. But that's what it's capable of, of doing ten.

Onscreen, you get eight faders. It looks like an audio console. The buttons that are on the console, on this virtual console, are all contextual. So if you have a microphone source, or, let's say, a hybrid, a phone hybrid or a codec that has a back-feed, if you touch an options button, the things that pop up are contextual. You don't have to ignore something or wish something else was there. The things that you need for that source and that type of source are right there.

So you touch the options button on the microphone, and you get options like auto level setting for the microphones, where you can take a sample of somebody's voice talking and the level gets set automatically. There's an automatic mixing capability for several microphones being open at once, and it will best mix the person talking and keep the others down just a little bit.

So this is really cool. Now, the actual engine, the part that does the actual mixing, well, that's separate. That's like more traditional broadcast gear. It goes over in the rack, and your audio inputs and outputs connect to it. It's a one rack unit device. Kind of low power. Does have two redundant power supplies available, and it's got some mic inputs, line level analog, AES inputs, also some AES outputs and analog outputs. So it's got a little mix of the things that you might need for a typical studio.

It also, optionally, it has RAVENNA, that AoIP standard, which includes AES67. So it's ready to talk to any other device of other brands that are also RAVENNA or AES67. So that's a real benefit there. You're ready to talk to other devices that come along as more broadcast gear becomes AES67 compliant.

All the usual things that you're going to find in an absolutely modern console, like support for talkback to guests. Talkback buttons automatically appear on all the channels that have mix-minus. There are precision stereo PPM meters. There's three mixing buses - Program 1, Program 2, and Record - and there's of course integrated CUE pre-fader listening. The mic pre-amps on it are very low noise. Really the best you can buy right there.

Also there's GPIO to turn, you know, tally lights on and off, or to mute some speakers if you've got to mute something externally.

It's the Lawo crystalCLEAR, and if this appeals to you, you ought to check it out. It's on the web at Lawo.com. That's spelled L-A-W-O. We always put their address in the show notes. If you want to go to GFQ or This Week in Radio Tech's website, you can always click on it there. Lawo, and then look for radio consoles, and look for crystalCLEAR. That's the name of it, crystalCLEAR.

There's also a video there. Mike Dosch, the director of virtual radio projects for Lawo, explains in quite some detail about how the console works. Check it out. Thanks to Lawo for sponsoring This Week in Radio Tech.

All right. It's our 242nd episode. We're here with Chris Tobin and Bob Holowenko. We're talking about, first, differentiated class of service with packets, so that some packets are more important than others. And now we're going to get in, in a few minutes, we're going to get into talking about how net neutrality might affect that. Because, you know, the way that net neutrality is proposed or talked about is that everything should be equal.

Now, does that mean everything or not everything? Should I be able to have a packet with a higher priority? I think really, the people who are in favor of net neutrality are really, they're not thinking about, or they're not considering, services that really should have a differentiated level of importance. I mean, do you really want your phone calls to have the same level of importance as your email? Well, they're very different things.

So, anyway, we're going to get into that in a few minutes. But before we went to the commercial, I asked Bob to consider and tell us about tools to help an engineer, a broadcast engineer or an IT person, understand how's my network performing? Bob?

Bob: So baselining is kind of the key to network performance. Or at least to understanding how your network is performing. If you've got quality equipment, like, if you're running enterprise-grade equipment, you can easily use things like SNMP tools, simple network management protocol tools, that'll watch interfaces and show you how much traffic is being used.

You can check for things like links that are saturated. Maybe you've got five floors of an office building, and all your data center stuff is on one floor, and you notice that between the rest of the floors, you've got saturation during certain times of the day.

Simple ping tests are great. I use a tool called MTR a lot, because it's both Windows and Linux-based - or it's available on both platforms - and all that does is it sends a bunch of rapid trace routes at whatever value you want. There's other tools, like PingPlotter and such. But it'll show you where there's loss within your network.

How else you can tell if there's problems in your network? Listen a lot to what users are saying. Obviously, you know, they tend to exaggerate things. But a lot of times they've got good information.

As far as seeing if it's up to snuff for adding new high-load items or specific items, one tool I use a lot - and it's a little more advanced - it's called Iperf. I guess it stands for IP performance. I assume it stands for IP performance.

What it does is it allows you to essentially craft packets and create packet flows. So if I knew that I wanted to do something that was maybe a 5 MB UDP stream - so just basically, just five solid MBs of data, and I wanted to classify it a certain way, I could use Iperf to test between two points to see if that's possible.

So putting it into broadcast terms, if you had a remote, and you had a cable modem on one side and your head office on the other, or maybe on two separate carriers, and you wanted to see if you could do, I don't know, 64k seems a bit low. So let's just say 512k for this remote solid back between the two sites. You could use a tool like Iperf and set it up for a couple hours and look at what your packet loss looks like.

Kirk: Ah.

Bob: And then if it's, you know, if it's something that is acceptable, go ahead and do your remote. If not, maybe you'll consider using a 4G LTE, or some other solution.

So it's a good way to tell if your outside network is fine, or your path to your network, and it's great inside the network. I use it a lot in troubleshooting wide area network bandwidth concerns.

Kirk: So the things that you mentioned, Iperf, MTR, PingPlotter, these are all PC-based software tools. Is that right?

Bob: Yes. They're available, well, Iperf is Windows and Linux. MTR is Windows and Linux. And Traceroute and ping is obviously Windows and Linux.

Kirk: Yeah.

Bob: And then PingPlotter is Windows only.

Kirk: Got you. Yeah, yeah. But what I'm saying is these aren't appliances. These aren't, like, a little fluke meter you plug in your network. These are, you know, PC operating system test tools, right?

Bob: Correct. I've used these before, actually, in those little Raspberry Pi units and other integrated computers...

Kirk: Oh, yeah.

Bob:... to actually make an Iperf appliance. Obviously you'd have to know a little bit, you'd have to be able to SSH into it, which means, you've got some control over the DHCP, or some other way of knowing its IP.

Kirk: Yeah.

Bob: But they're great little units that are, they're okay up to about 50 MBs. But for a WAN, for a wide are network, that's really all you need. It's easy enough to throw something like that at a customer site, make sure that the port forwarding allows you to get the port through, and boom, you can do an Iperf a few days ahead of a remote, for instance.

Kirk: That's a great idea. Now, a lot of the tests that you mentioned, like just doing a ping or doing a speed test, I've always thought, well, that's a great place to start. But, you know, doing persistent pings it doesn't tell you how much throughput you could get. It does tell you if a ping gets there and back and if it's dropped. So if there is saturation going on, the ping might likely get dropped.

So I've always thought that these are kind of proxies for what you really want to know. What you really want to know is, "Can I send packet after packet after packet after packet, and have them arrive pretty reasonably on time, pretty reasonably in order, at the other end, at a given bandwidth?"

What you just mentioned about Iperf, it sounds like Iperf will do exactly that. You can almost simulate the packet flow that you want to actually have at some point in the future.

Bob: Exactly. Using something like a SpeedTest.net, like, the Oopla suite of tools, it's a great way to know how your network was performing about 10 seconds ago when you ran it. But that doesn't tell you anything over time, or it doesn't really give you any kind of reasonable expectations.

The other thing, too, is that Oopla runs up to eight simultaneous connections. It's got features where, the way that it works is it'll do multiple connections in order to saturate a link regardless of latency. And I realize we're starting to get into some heavier stuff here. But it creates eight sessions. Whereas a lot of other things are single session.

So, for instance, if you're out with a VoIP phone or, I'm picking on VoIP a lot here, because it's very specific to high-quality - you need high-quality for it. A VoIP call is a single session each direction using real-time protocol. So that means that if you get one packet missing from a real-time protocol stream, you're going to notice that a lot more than one packet in one of eight PCP streams that are all fighting with each other anyway. So if one loses a packet, the next one's going to pick it up.

So you can sort of get false comfort, I guess, from a good speed test result.

Kirk: Hey, I'm going to ask...

Bob: Yeah?

Kirk: Bob, I'm hearing a little bit of that funny noise. You might want to... is your audio device USB connected, Bob?

Bob: Yeah. It's a Blue Icicle.

Kirk: Tell you what? Why don't you just try unplugging it and plugging it back in? I'm going to talk to Chris for just a minute.

So Chris, when you show up on-site at a customer's location, and you want to consult with them as to, "Hey, can I do this - I've got to do this remote. What do I need? Is the stuff I have now going to work? Do I have to order something else?"

What are some of the first tools that you'll bring to bear in that situation?

Chris: First step, I sometimes will bring with me with, on the laptop, I'll have, like, a 10 MB file that I'll do an FTP transfer between that site and my FTP server at the office, and see how that transfers. I'll use the SpeedTest.net as a quick, let me see what's going on. And Bob is right. You get a lot of false readings, thinking that everything is great when it's not.

I get a look at that, and there's this little graph they have on it. Believe it or not, that little graph is relatively indicative of what you're going to expect.

And then I use, what's that other program? There's another website that I use. I can't think of the name of it now. It's a VoIP testing website and that seems to be the most accurate that I've been able to find, that works the best.

I haven't used Iperf in a while. There was another SMP tool that I was using. But the software I had, I got it from somewhere, and it's no longer, doesn't work on Windows 8. And I was like, forget it. I'm not going to go crazy trying to make it work.

Those are the things I do real quick. But I do a VPN from the laptop back to my office. I see how it performs that way. There's a few things I do. Because, as Bob pointed out, you know, a lot of the tests you do, you have to work of do in real-time and get a baseline over time, rather than just a snapshot for the 10 seconds you're connected. And that's why I do various different tools. It's...

Kirk: You know, that reminds me of something, Chris. You know, back in the ISDN days, which we're at the trailing end of right now, hey, you had an ISDN connection that would drop out from time to time, and you weren't getting good audio all the time.

Chris: T-bird.

Kirk: So you asked the telco, yeah, you asked the telco to come out and test this ISDN. And they put a T-bird on there, or whatever, and they check it, and they do a bit error rate test, and BERT, for, you know, a minute, 15 seconds.

Chris: Yeah.

Kirk: However long it lasted, you know, long enough for the telco tech to have a smoke, right? And it'd check out fine, and he'd pack it up and leave. But, and then in the middle of your NFL remote, you know, it starts dropping out again.

Chris: Yeah, yeah.

Kirk: So that's why we advise people to, no, make the guy stay there, or tell him to get lunch and come back. You'll watch his gear, yeah? So...

Chris: Well, you know, if you did a lot of T1 work back in the day, the smart jacks and the QRSS tests that you would do for BERT testing, certain phone companies at the time, they would require a 24-hour test. You know, if you said, "Look, I'm having a problem with my T1s. I'm not sure what's going on. We're getting dropouts, slipped bits, and things are happening. Can you do a test?"

They would go into the Smart Jack and monitor it and run a 24 hour test. Not a test test. They'd monitor it 24 hours to see it full over time, and if the circuit was really in trouble, and they brought it back up during re-engineering, as they would call it, then they would run the tests for 24 hours.

Nowadays you're lucky to get them to do, I think, 10 minutes. But, you know, that's what they were doing, baseline over a 24-hour period. So they could tell you, or look at your engineering and say, "Oh, wow, look at this. At this time of day, there's been some issues." Then they look at their network and say, "Oh, our network was very busy at that time, as well."

I can tell you, when I worked at a network, one of the broadcast networks, the carrier we were working with, AT&T, we would have issues with certain T1s across the country. After a while, they started doing analysis from their Network Operations Center. The GNOP, the Global Network Operations Center.

They discovered they had a bad span. A bad DS3, or I forget what they called it at the time, and they noticed at certain times, when traffic would increase, the span would start to have trouble. The endpoints would have trouble.

It was all, you know, just one of these things, if they hadn't done a 24-hour test on one of our circuits, because we had a service level agreement, three different, three sets of virtual T1s from New York to Dallas to LA for the programming distribution. So we had a high-level, you know, enterprise account.

We found a problem. And we were, like, all sitting in the room going, "Wow, that's interesting."

The problem, the span that was in trouble, was somewhere between Chicago and Kansas City. To give you an idea of how they were routing the digital packets.

Kirk: Yeah.

Chris: So yeah. I mean, you go out to do a remote these days, outside broadcast, your best bet is to try and do it over time. If you, you know, do it a couple days before and really look halfway at the information you're gathering, I think you'll be in great shape.

I've done it on many occasions at sports bars, where I've done some remotes recently. And we lucked out, made it work. Despite the fact that the guys on-site were like, "Wow, we failed to get the email to work. How'd you guys do that?"

Kirk: So let's move on to this notion of net neutrality. However you may feel about that. This person, the Networking Nerd, writes just last November - and this is the US. I realize Bob is in Canada, so I'm going to ask Bob how this may or may not apply in Canada.

"The president...", and the blogger was speaking of President Obama, ". . . recently released a video statement urging the FCC to support net neutrality and ensure there will be no 'pay for play' access to websites or punishment for sites that compete against a provider's interests.

"I wholeheartedly support the idea of net neutrality. However, I do not... I do like to stand on my Devil's Advocate soapbox every once in a while. Today I want to show you why a truly neutral internet may not be in our best interest.

"If the FCC mandates a law that the internet must remain neutral, it will mean that all traffic must be treated equally. That's good, right? It means that a provider can't slow my Netflix stream or make their own webmail service load faster than Google or Yahoo. It also means that the provider can't legally prioritize packets, either.

"Think about that for a moment. We, as network and voice engineers, have spent many an hour configuring our networks to be as unfair as possible. Low-latency queues for voice traffic. Waited fair queues for video and critical applications. Scavenger traffic classes and VLANs for file sharers and other undesirable bulk noise. These plans take weeks to draw up and even longer to implement properly. It helps us make sense out of the chaos in the network.

"By mandating a truly neutral net, we are saying that those carefully marked packets can't escape from the local network with their markings intact. We can't prioritize voice packets once they escape the edge routers. And if we move applications to public cloud, we can't ensure priority access. Legally, the providers would be forced to re-mark all class of service and DSCP values at the edge and wash their hands of the whole thing.

And what about provider MPLS circuits? If the legally mandated neutral provider is administrating your MPLS circuits (as they do in small and medium enterprise), can they copy the DSCP values to the MPLS TE field before forwarding the packet? Where does the law stand on prioritizing private traffic transiting a semi-public link?"

Now, I don't understand about 30% of what I just read. So Bob Holowenko, I hope that you'll step up to the plate and explain a bit about this person's concern.

Bob: Right. So what he's saying, essentially, is that inside the corporate networks, inside our local area networks and enterprise networks, well, we're essentially doing what we don't want the carriers to do. We're giving certain classes of traffic priority. In this case, he uses voice, as we have throughout the discussion so far, talking about low-latency queuing and, yeah, fair queuing for video, and then scavenger classes for everything else. So your bulk traffic.

And that's sort of what we're making, we're asking the government not to do. Or, you know, the internet community as one is asking the internet community not to do. Here he's saying that there could be some benefit by allowing VoIP packets to carry over their values.

So let's break this off a bit here. At first, he talks about how, when it's put into a carrier network, sorry, into an internet service, all the DSCP values, these different tags that you assign for QoS, are stripped off.

And yes, that's the case. Even the company that I worked for, we strip off all tags as a packet comes into the network. Because what's to prevent somebody from marking all their traffic as high-priority as it leaves their router, and then it unfairly goes out into the network ahead of your neighbor's traffic? Sure, I'd love to have that for mine, but it's just not fair.

So there is some merit to what he's saying. I don't necessarily agree. I think that what we should be looking for, if there is any kind of regulation that goes on, we may want to be able to have an additional service where we say, "for VoIP traffic, for it to be able to have precedence, we'll pay an extra $2." That's a slippery slope.

Kirk: Yeah, yeah.

Bob: A very slippery slope. So what we would need is some way to have the carrier and the user agree that certain traffic is VoIP. Well, there's nothing really to prevent me from putting out all of my traffic to make it look like it's VoIP.

Kirk: Ah.

Bob: So then they'd have to create something like, okay, we'll give you 64 kB per second of precedence of EF traffic, but if you exceed that, it gets marked down to regular.

Kirk: Yeah.

Bob: Or they'd have to come up with all these complicated ways that are just not any good.

So what he's saying is that we're creating something today inside our network that we're asking to not exist in the internet world.

Now, when you start looking at the other thing he brought up was MPLS circuits. And this is something that I spend, well, that's my bread and butter, is consumer wide area networks. Particularly carrier class networks like MPLS, VPLS, even old frame relay services over T1, or fractional T1, or original friend relay, twisted pair.

Those types of services, he says, may also have the same kinds of problems. And I'm going to say that's not the case. I would be very surprised to see an MPLS provider - so a wide area network provider - mark traffic - or, sorry, have the MPLS traffic be at the same level as a consumer internet connection.

So what we need to make sure of - us, as the consumer - well, sorry, us as the business consumer, if we're talking about, you know, a broadcast engineer that's got a wide area network that runs on MPLS, is we would want our traffic, our MPLS traffic, our WAN traffic, to be protected.

So if there's common infrastructure shared between an internet router and an MPLS router and I'll let you in on a little secret. A lot of providers are starting to use the same hardware for both, and then on the trunks between the equipment, they do their own value system. They call it MPLS TE that he references here.

Kirk: Ah.

Bob: And that stands for MPLS traffic engineering, which allows you to essentially map a DSCP value to a TE bit value. Or, sorry, a TE value field value and that's rather complicated, but it might be the answer to protecting MPLS traffic.

I guarantee you, all the vendors out there are doing it already. Sorry, not vendors, but carriers, are protecting that traffic already, because there's a lot of saturation on the internet today. And if your MPLS was like that, it would be a lot different.

We do need to draw a line between what is an internet service, what is a WAN service, and what is a tariff service. And I'm of the belief, and you know, I speak as me, not the company that I work for, or for the company that I work for, because I know that they would strongly disagree. But MPLS type services, we might need to start looking at a situation where they're tariff-based like an ISP, and like a frame relay, like a lease line, if we start running into these issues where you've got internet traffic contending with MPLS traffic.

But for the time being, we should think of them as two completely separate things, which is why I sort of disagree with the author.

Kirk: Wow, that's a lot to digest. What you're saying certainly sounds reasonable. But that, well, this is an argument that's a bit like health care. There are so many facets, so many interested parties, that it's a little bit to see a clear path to a lack of chaos. And I think that's one thing that bothers a lot of people, is the chaotic nature of an unregulated free market.

Hey, I'm sure that Chris Tobin has some things to add about this, and I want to hear what Chris has to say. We're going to take a quick break right now, though, and hear from one of our sponsors.

You are watching or listening to This Week in Radio Tech. It's episode 242. I'm Kirk Harnack, along with Chris Tobin in New York and Bob Holowenko somewhere in the middle of British Columbia, where he barely gets two internet feeds.

Our show is brought to you in part by the folks at ProSTREAM, which is kind of a new brand at Telos. And Andrew, if you're ready for a message, we've got a message from ProSTREAM. It's just under two minutes long, and it's a lot of information packed into a very short period of time. I know because I wrote it.

So if we could have your undivided attention for a minute and 45 seconds, I think you'll find this very interesting? Andrew, roll it.

Video: Streaming audio? It's everywhere. Wi-fi and connected cars mean easy access to your streams - or your competitors'. You need the right tool to grab the advantage. ProSTREAM X/2 streaming software from the Telos Alliance, designed for stations whose audio quality is just as important for streaming as over the air.

With ProSTREAM X/2, you get premier audio processing from Omnia Audio, the same used by top radio stations around the world. Input audio from sound cards or via LiveWire and RTP stream. And for ultimate audio control, upgrade to the ProSTREAM 9X/2, adding Omnia 9 audio processing with Undo technology to correct overprocessed source material.

After processing, the ProSTREAM X/2 will encode streaming audio in multiple formats simultaneously, sending to multiple destinations. Choose from MP3, AAC, and popular AAC family members like HE-AAC V2. All coding algorithms are fully licensed reference code for top quality sound.

ProSTREAM X/2 is ready for adaptive streaming, another streaming feature separating you from everyone else. With Multi-Rate Adaptive streaming, you and your listeners are freed from manually selecting the tradeoff between connection stability and audio quality.

ProSTREAM X/2 features control via HTML5 graphical web interface and REST API. It's Cloud-ready and can be run entirely off-site at your data center. Plus, SNMP is fully supported.

ProSTREAM X/2 - Professional Streaming - from the Telos Alliance.

Kirk: Ah, what a fabulous commercial.

So thanks a lot to ProSTREAM for sponsoring This Week in Radio Tech. You know, as streaming becomes more and more popular, you need ways to make sure that your stream sounds really good. It used to be we could take an old cast-off audio processor and stick it in front of some freebie software encoder and call that our stream, and that's just not so good anymore.

So do it right. Check it out on the web at... go to TheTelosAlliance.com, or I should say TelosAlliance.com, and look for ProSTREAM under both the Telos and the Omnia brand names for now.

All right. You're watching This Week in Radio Tech. I'm Kirk Harnack, along with Chris Tobin and Bob Holowenko. Chris, comments about what we think we might know about net neutrality? Of course, we really don't know a lot yet with what the FCC's going to come out with.

Chris: Well, I'm not a fan of the regulation part. I still think that we leave things as is and let our markets do their best. As Bob pointed out, there are many ways you can, you know, create networks and connections. I've worked with a couple of ISPs who are not the traditional legacy, you know, Bell operating companies that are out there, from Telos to, what do you call it? Rogers. Not Rogers. Bell Media. And then here in the States, folks with Verizon and AT&T.

The guys I've worked with, we came up with some really nice connections, fair rates, and everything worked out just fine. But based on what people are trying to do, it's like, "Okay, I'm going to lose a lot of opportunity, because now the regulations are going to say otherwise."

Here's why I say this. Working with this independent ISP, we were talking about doing ISDN stuff. They had access to it. Jis first words out of his mouth was, "Well, yeah, sort of, but the tariffs limit what we can do and how we transfer traffic, so we don't bother with it anymore." And I was like, "Oh, interesting."

And there was a few other things that I'm not going to get into, because they were, like, upset when I started asking them questions. A lot of the tariff stuff that we're familiar with now, they can't access because they're not part of that, you know, tariffing common carrier clause and several other things.

So I'm thinking the best way to the network issue, as far as whose packets go where and who does what, should be left up to, basically, the end user.

If you're a business or a company or corporation that has tagging on your network and you're doing it yourself, well, that's within your business. That makes sense. You shouldn't expect it to go out onto a public network connection and extend your stuff. And if you want to extend it, then you go to somebody and get an MPLS and VPLS and say, "I'm going to extend it to my other business office, business units, whatever, and here's what we're going to do to extend the network."

But I wouldn't put the onus on, you know, the current ISPs that are out there and just say "Hey, you know, this is what we want to do." I mean, I'm just not a fan of regulation stuff. Working in broadcasting for 30 years, and some of the stuff I've had to put up with to try and get things done because of the forced regulation.

Then the phone companies who are sort of taking advantage of the fact that, "Well, sorry, can't help you. According to the tariff, we can't do that." I'm like, "Yeah, the technology can do it."

"Yeah, we can't do it. But if you want to, we'll charge you $10,000 and make it happen."

"Uh huh." I'm like, "Well, I can go somewhere else."

"Nope, you can't, because they have to come in through our cables to do that."

See, that's what's going to happen. That's the unintended consequence that's going to occur after we start telling people, "Here's what you do. Do this, do that." And, you know, Comcast gets what they want.

Kirk: I have to agree with that, that when you regulate anything, the lowest common denominator, the least common denominator, becomes the standard of service.

Chris: Yes.

Kirk: The worst it can be becomes the standard and you lose the incentive to innovate.

What's messy about a lack of regulation is the chaos. "Hey, you know, I only have one internet provider available at my house, and it hasn't been regulated. I should have two. Or I have only one choice available at my house, and it's not a good choice, so I really only have one choice. I can't vote with my dollars somewhere else."

A lot of folks are much more afraid of or concerned about that. And "folks" I mean both consumers at home...

Chris: Mm-hmm.

Kirk:... like I am right now, and professional applications. Big businesses, small businesses, businesses who depend upon internet communications in order to make things work.

What I also find interesting is that, I realize that my radio stations are small businesses. We use all consumer-grade internet for everything that we do that needs internet. You know, it actually works just fine for us. I can imagine having to pay a whole lot more for, I don't know, some greater class of private WAN service, and not really getting much better performance.

You know, the weakest link in our operations is our link to American Samoa. And you have a situation in American Samoa, hey, I think Wired magazine called American Samoa the most expensive internet in the U.S. Uh-huh. That's true, it is. We used to be paying about $500 a month for one tenth of a 512 kilobit connection. So, yeah, it's the most expensive internet connection that you can have in the U.S.

Part of the reason it's expensive is because in Samoa, it's all government regulated. It's completely regulated. You can't get internet, well, I guess, yeah, there is a competing service that's come along, but they're highly restricted by having just one satellite.

So, man, you know? I get your concern, Chris. Because if you regulate it, it ties people's hands. There's no incentives to do better than the bare minimum.

Hey, we'll throw back to ...

Chris: Well, you know ...

Kirk: Oh, I'm sorry. Go ahead, Chris.

Chris: Well, I was going to say, when you said that you have no choices as far as what you can, you know, use elsewhere and whatnot, part of that is because of the current regulations in place. Take, for instance, where I am at. There are three different cable companies/ISPs available in the area. However, the building I'm in can't get any of the other two, because one has decided they're not going to expand on their fiber anymore. They made that clear at a shareholders' meeting several years ago, and the other one can't get the proper licensing because of the tariffs and the way they are regulated.

So it's only left to one that comparably can get what they want because, oh, they were the first ones here in town 20 years ago.

So, you know, net neutrality will only just make that worse. Because then if you force that other company to bring in that fiber connection to the building, they will say, "Well, we'll do it, but we're being told to do it, so this is what it's going to cost, and here's what we're going to offer, because we can't afford to do anything more."

I think if people, more people, you know, tried, forced the hand of others that you'd see. Like, say, for instance, here in New York City. On several occasions, there have been proposals for wireless municipality internet. Who are the number one companies that fought it and said it was discriminatory?

The phone companies. How is it discriminatory that a city decides they're going to give you, 1MB bandwidth down, 512 up? Nobody seemed to fight it. They just let it go, and it's gone.

Now, and things like that, that's what's going to happen. You're going to have more of this, because these guys don't have to do anything because they'll sit back and go, "Well, we're regulated now. Sorry. Can't help you."

There were several wireless loop companies here in town. One of them is in Brooklyn, that's been trying to provide local internet service to people, and they're getting slammed by the incumbents, or the legacy guys saying, you know, "This is an unfair practice."

"Well, how is it unfair? They're going to you for the internet service. They're just providing a different means at the last mile."

That's what's going on, and nobody's arguing.

Kirk: Yeah.

Chris: You know, you don't need net neutrality. It's a load of bunk. What you need to do is focus on what's been tariffed and regulated that's all out of whack, because it favors only a certain group.

Just like in American Samoa. I'm sure there's transit fees and all kinds of other stuff that are being regulated and tariffed, and that's what's forcing the pricing. Or, because they don't have to do anything else, they can price as they like.

Kirk: Here in Nashville, sometimes I'll put a little post on Facebook or Twitter about this. I mean, I guess every two or three years, I end up making a comment that my Comcast service just keeps getting faster and costing the same. Or I think they've bumped it up $5 in the last eight years. Google has been threatening, if you will, to bring GB fiber to Nashville.

Well, I'd love that. It's funny how Comcast starts getting better and better, and my TV rates went down, when Google started making noise about bringing fiber into Nashville. Competition's a really good thing for consumers. I don't know if it's good for shareholders of large ISPs.

Maybe, Bob, I don't know if you can speak to the business end and the motivations of different ISPs, but maybe you can provide us with a little clarity on how we might think about this.

Bob: It's hard to step into this without getting into a bit of a rabbit hole.

Kirk: Ah.

Bob: But there's a lot of factors. Well, if we talk about the price of delivering a service, and how in some areas it's going up, in other areas it's staying the same. A lot of it has to do with things you wouldn't necessarily expect, like geography and delivering a service.

I mentioned in the preamble to the show that I do have issues with my internet connection here, and I have no other options. So I'm stuck. Whereas, you know, it sounds like you could go to Google if you needed to. So for them, for Comcast, there is some influence, some pressure on the business side to make that service better.

Where I am, I'm trapped. The cable company that I use has horrible saturation problems between where I am and the next large center towards the south. And it's been an issue for about six months. It's been nonstop calls to them. But there's not a lot of pressure on them immediately, because the other carrier here, the DSL provider, doesn't have much of an outside plant as far as DSL capable. There's a lot of copper around, but it's old copper. The seal equipment for DSL is only seal equipment. It's not remote equipment.

So the cable company here hasn't had a lot of pressure to really compete. But now that company that owns all of the DSL in town realizes that their infrastructure is "horrid". Their terms, not mine - their explanation, not mine. So now there's trucks all over my community through GPON in. So as soon as the GPON started going in, every time I called the cable company to say, "This is a problem."

"Oh, yeah, we're working on it right now. We've got techs engaged. It's happening."

Well, what they don't know is that I know people in that company, and they're saying, "Yeah, it's a little bit slow. We'll admit that there are some problems getting this going."

So on the business side, it's all competition pressures. If there's no competition, they're not going to make any improvements. Now that the cable company has pressure from somebody who can deliver a much faster and less latent service, all of a sudden things are happening. Those squeaky wheels are starting to get the grease they need.

Chris: That's true.

Kirk: That is true.

Chris: That's why DSL hasn't improved.

Kirk: Yeah, that's true in just so many ways. Oh my goodness. If you're the only restaurant in the whole mall, you know, you don't have to be that good and you're going to get most of the mall business.

Chris: Yeah, it's true. Yeah.

Bob: One of the guys in the chat actually made a very good point. Brian Monroe mentioned about how the way right of way is done in a lot of places makes it quite unfair, nd that's very true. Well, actually, here's the best example. New building complexes are being built, and one of the incumbents - either the cable carrier or the phone company - they're all now working on the same space. They're all doing the quadruple play. Well, triple play, quadruple play, with phone service, internet, and television service.

One of the problems that we're having is now that there's so much competition in all three, because everyone wants to bundle something, there's certain buildings where the building developer is signing contracts with one of these providers to be the exclusive provider for that building. So now everybody in that building doesn't have the option to choose the other company.

Which means that if for an ethernet to the suite or for a service, if you're not happy with that provider, you're stuck. So they can continue to raise prices, as the terms of agreement say. It's creating a lot of people unhappy with a couple of cable companies in this area, and now the DSL providers doing the same thing.

Kirk: Yeah...

Chris: Yeah, that happens a lot.

Kirk: ...That's very disappointing, that building owners would make an exclusive deal. Of course, they... you know, they're getting their palms greased. "Hey, we'll pay for all your wiring in the building, but you've got to make us the exclusive provider."

Chris: And, well...

Bob: There's a lot of leverage, too, that when you've got phone companies that are also mobility providers, they'll also bid on roof space. So there's a lot of leverage out there.

Kirk: Wow. So let's keep this back on broadcasters. I guess it is too early to speculate. When people who want net neutrality, okay, let's talk about that motivation for just a minute, before we have to close out. I'm guessing most people want net neutrality because they don't want their Netflix interrupted, like it was over Comcast and Verizon for several months. This was about a year ago.

A year ago, I was living in Netflix hell. Netflix, half the time, didn't work. We just about canceled it, because it was being throttled terribly through Comcast and through Verizon.

Say what you want. They tried, you know, various... seems like Comcast and Verizon wouldn't play like the other ISPs would, where Netflix puts servers on Comcast's network, and puts servers on Verizon's network, like they had on other networks. Netflix doesn't just have one set of servers. They've got servers on other people's networks so that they don't have to cross all the, you know, peering points. But, you know, Comcast, Verizon throttled it, made people very unhappy. I forget what the whole upshot was, maybe somebody can remind me.

But that spat, I think, gave the net neutrality proponents a lot of encouragement and muscle and forthrightness in saying, "No, we can't have this happen. We need net neutrality." But boy, you think about all the consequences to that, if everything really is treated equal. I don't think you'd want that. Would we be better off had Comcast and Verizon not played nasty with Netflix?

Chris: Well, they only played nasty because they want to double dip and make some extra money. I mean, you know, I have Time Warner Cable.

Kirk: Yeah.

Chris: And I can tell you on many occasions, watching Netflix, when suddenly it's buffering and suddenly stops, and drops the connection. I'm like, "Wow, that's interesting." Then as the infamous box itself, the box is IP-based box from Cisco, just decides to reboot itself on its own about four times a day. You know, there's all these things that are going on which makes me wonder really if the networks are properly being managed.

Also, that Netflix issue with, I think, Comcast and Verizon and stuff, I think that was actually more back-office with the transit fees, and what they were claiming and costing the handoff. And, you know, they just basically want to double dip. They charge me for internet service so I can connect to Netflix or Google, then they go to those guys and say, "Well, we're going to charge you to connect to my network to get to your customer." It's like, "Hello? Nobody's arguing that?"

Also, as far as the buildings doing exclusive deals, that's been going on since the beginning of time. If you have smart people in the building, if you have, say, a condo association or a homeowners association, the smart thing to do is the board should be thinking of options of keeping everything open, and not just being narrow-minded or single-sided and say, "Oh, yeah, we'll just do this, and they'll put an antenna on the roof and subsidize the installation."

"Great, thanks."

That is a recipe for disaster. I've been on the board of two buildings I'm in, and we had several times the local common carrier, the cable company, come to us and say, "Well, we'll do this and this, and do it exclusive." I'm like, "No. If you want to share the space, the crawlspace with the other guys, more than happy to, and we'll offer up, you know, whatever apartments want to connect. Otherwise no, you're not going to get exclusive."

And you know what? As soon as we said no, all of a sudden the fees were lower and the cost was a deal. We're like, "Oh, look at that." You know?

Then there are other buildings I've been in where we've got 500 units, and the local Fios company said, "No, it's not worth the trouble for us to bring fiber into the building."

"Really? You brought it into the building across the street with 96 units. What's the difference?"

You see? That's why I'm saying you don't want the regulation. You just want the market to say, and these guys - we'll call them "these guys" - will have to fend for themselves.

Just like Bob said up by him, he's getting, you know, his local people are getting competition. Yeah, it'll take some time probably for everybody to realize, "Uh-oh, we've got to up the game." But that's what you want. That's the best way it's served.

That's just my opinion, though. I'm just saying.

Kirk: Yeah. Competition and the free market are a bit chaotic, and that makes people on fixed incomes, for example, nervous. And people who, you know, I get that. I don't agree with it, but I...

Chris: Good businesses would look at that and say that's an opportunity to work something out.

Kirk: Yeah.

Chris: That's how you do it. You don't say, "I'm forced to do it by federal mandate so, you know, here's your reduced fee. I'll give you cable service for $50 a month, but you'll be on the cable connection that's rusted and not working well and, well, if it goes out, we'll get to you eventually." That's what happens. Because I know two people and family members who are on fixed incomes who are retired, and are fighting right now with the cable company because of that.

Kirk: Hey, Bob? I've got a question that's still burning in my mind here. I guess one final question before we're going to have to take our last break and sign out. You've mentioned a couple times saturation.

Bob: Mm-hmm.

Kirk: I want to see if I can get a little better understanding of where saturation occurs and obviously, why, because more packets are trying to get through than the bandwidth allows. But and I guess I want to understand why isn't saturation a reasonably easy problem to fix? All it takes is throwing money at it.

Now, if it takes a new run of fiber between two locations, and that's going to end up costing a million dollars, I get it. That doesn't happen overnight. But if it's a matter of putting in, you know, some more cards at a peering point, and some more patch cables, then I don't see why saturation is such a problem. Help me understand why it's difficult to solve at the ISP level.

Bob: Well, the first part of the question was where can it happen?

Kirk: Yeah.

Bob: And it can happen anywhere along the link. So most common place that we see saturation is in provider to provider links. And so this is between two companies. So this might be between somebody like a Comcast and somebody like well, let's use a third party, like a Tata. But they're going to have lots of distributed links. They're going to be using EGP to be doing all kinds of things.

So they can move those around. Those ones are easy to fix, because it is just throwing money at it. It's buying more cards. It's linking or unlinking aggregated sets of fiber pairs. There's lots of things that could be done to fix that in the near term.

Where it's more expensive is when it's something like a concentration node to, let's use cable as an example. When you've got a bunch of modems on there, all the DOCSIS modems are on one common piece of coax.

Kirk: Yeah.

Bob: They're on one common broadcast medium. So if you've got one neighborhood that's got 64 people on the link, and you've got one neighborhood that's got 150 people, well, that 150 people, their combined traffic could be enough to saturate, or fill up all the time slots, I think DOCSIS is actually CSMBCD. But enough traffic to make it so that your modem has to wait for that next push out, or is waiting a longer time for the next packet testing for it.

So that's one place it can happen, and that's at the local user. The next hub, or the next place it can happen - and this is what I'm suffering from - is from that central point where either all the cable connects or the D sign where all the DSLs connect, up to the next hop into the core of the network. So you see a lot of saturation there due to increased user demand, like Netflix or YouTube, whatever.

Then from there, it's into the core, the carrier core, then the other hop that I talked about, where it's provider to provider.

So there's lots of different places where it can happen. Some of them are cheaper to fix than others. Some of them are faster to fix than others. Usually the closer the issue is to the customer, the more expensive it is going to be to fix.

Kirk: Ah, okay. The closer to the customer, the more expensive, because it involves more plant infrastructure?

Bob: Exactly. We there is some caveat, though, too. Like, here in Canada, we've got a problem where peering with American companies is harder because they won't come to us. We have to go across the border to them. Which means we're paying for much more expensive links. Usually, you know, anything that crosses that 49th Parallel has the "49th Parallel tax", we call it.

Kirk: Ah.

Bob: Where it just instantly is way more expensive. So we have to buy facilities from somebody who's servicing along the border. They have to then interface with us. We use that to connect between us and other carriers. So there's a lot of effort that goes into that.

That's not just the company that I work for that has that problem. It's other ones. So when you talk about these other solutions, like how Netflix puts caching services in other providers' networks. That's to eliminate the amount of bandwidth needed between Netflix in, I don't know, let's say California, and the user in wherever. Because now you've got something caching in between, and if something's popular, it's going to be held.

When you go to something like Facebook, the majority of the non-dynamic content is actually doing that. It's cached, because it's downloaded multiple times. If you watch something popular, it's cached, and it's probably going to buffer faster.

Kirk: Ah, yeah. There's that. Okay. Okay.

Hey, we're about almost out of time. We're going to come back in about three minutes.

A final word on this issue, net neutrality, broadcasters and the internet. Because we're all very dependent upon that now.

Hey, one of our sponsors is Axia Audio. And Axia's just started shipping, actually, before the holidays, the new Fusion console. And my friend Clark Novak - this guy just can describe this thing much better than I can. So Andrew, if you're ready, let's roll the tape and see what Clark has to say about Fusion.

Clark Novak: Hi. I'm Clark Novak from Axia Audio, and I'm here to introduce you to the new Fusion AoIP mixing console, the newest modular AoIP console from Axia, the company that invented AoIP for broadcast in 2003. Let's take a quick look at some of the unique features found only in Fusion.

After 10 years and more than 5000 consoles, people constantly tell us how attractive Axia consoles are. But a console isn't designed for show, it's made to work in challenging conditions, 24 hours a day, year after year. And here's a look at some of the special design choices Axia has made to ensure that Fusion meets that challenge.

Some companies cover their console work surfaces with paint, which can rub off, or with plastic, which can tear or be ripped. Not Fusion. Its work surface is all metal, solid aluminum. Not only that, its double anodized markings are sealed in. They can't ever rub, peel, or flake off, which means the Fusion will still look as good in five years as it does the day you begin using it.

At one time or another, we've all had the task of replacing light bulbs in console switches. Fusion does away with all that. All switches are lit with LEDs, made to keep on shining for hundreds of thousands of hours. Oh and those switches themselves are aircraft-grade, specially sourced and tested by us to sustain millions of on/off operations without failure. So you won't ever have to worry about replacing those, either.

Fusion's frame is made from thick, machined aluminum, too. It's RF-proof, but also lightweight. No worries about whether your tabletops can hold up.

Fusion's designed for drop-in installation, and it's very low-profile. No giant tub to intrude on under counter space.

Whether other consoles use dot matrix readouts for channel displays, Fusion comes with easy to read, super high-resolution OLEDs above each fader. They show the assigned source, tallies when talkback or other special features are enabled, and full-time confidence meters to help prevent dead air. Talent doesn't have to wonder whether that caller is dropped or a satellite feed's ready to join. They can see it clearly before they pull the fader up.

No wipers to wear out on our rotary encoders. They're all optical.

Some of the most important parts of any console are the faders. One of the reason faders fail is from dirt, grime, and, of course, liquid that falls through the slots of the modules. Fusion's faders are special premium conductive plastic faders that actuate from the side, not the top. That way, dirt that falls through the surface slots falls past the faders, not into them. They stay smooth and silky nearly forever.

That's a fast look at how Fusion consoles are designed to last, and built to perform just as beautifully as they look. We'll see you next time.

Kirk: I want to thank Axia and the Fusion console for sponsoring, being one of the sponsors on This Week in Radio Tech. Check it out at AxiaAudio.com. It's shipping now, and it's gorgeous.

All right, Bob Holowenko. There's one more thing we want to talk about, how net neutrality - or, you know, saturation - affects broadcasters. And you said streaming. Tell me about what we ought to worry about.

Bob: A lot of stations are putting a lot of emphasis on their internet stream, because they may get a large number of listeners through that, and now that a lot of people are doing the PPM marketing, and there's all these other methods that they're using their stream for, additional ad sales, etc. If there was a situation where we went into a non-neutral internet environment, or where there was no regulation, the listener experience for somebody on the web stream could be reduced a lot, or they could end up in a pay-to-play situation where, you know, if they want access to all of the tune-in streams or whatever, somebody like a Comcast or another provider may say, "Okay, well, we'll give that to you at a high quality. We'll give you that on the fast lane", which is a term that's been coined a lot for this, "for an extra $5 a month." Or, "If you want the streaming media package, that's $10 a month, and you have to get everything else with it, like the iHeart radio," or whatever.

So those sorts of threats exist for broadcasters, because that's a lot of revenue that could be thrown out, or a lot of people who have to pay to get to something where you want high listenership.

Kirk: Got you. Well, we'll have to cover all this on a future show, because we're about out of time. I guess what people worry about is having to pay extra to get the things they want. But now we see cell carriers saying, "Oh, you want to stream music? Well, if you choose this one or this one, we won't count it against your data."

Now, some people might say, "Well, that's unfair." Well, it's better than it used to be. If you don't choose one of the streaming services that the carrier's associated with, then it just costs the normal data usage that you used to use on your cell phone. But certain ones are free.

You know, on the one hand, I don't have a problem with that, but I can see how it'd make people nervous about what may be coming down the road.

Chris Tobin, any last comments?

Bob: It encourages them.

Kirk: Yeah, yeah. Chris Tobin, any last comment?

Chris: No, no. As far as broadcasters are concerned with the way things are going with bandwidth and networks, best thing to do is just learn how to build a robust infrastructure, and you go from there. As far as tariffs are concerned, try to avoid them, and vote them down when you have the opportunity.

Kirk: Hopefully, to serve broadcasters' needs, someone will always be standing by to provide the service that you want and take your money for it. That's kind of all that consumers and professionals are asking. "Here, take my money, but give me what I want."

All right. You've been watching This Week in Radio Tech. Bob Holowenko, thanks for staying the whole time. You know, I asked you to make a cameo appearance, and, you know, here you stayed on the whole time. I appreciate that. Hope you didn't cut into your evening.

Bob: Oh, it's fine.

Kirk: Okay. All right. Good.

Chris Tobin, and thank you for being with us also for the whole hour. I sure appreciate that.

Chris: No problem. By the way, check your email regarding what I was offering up for a theme as a last resort.

Kirk: Oh. I'm sorry. I haven't checked my email in a while, since the show started.

Chris: That's all right, no. Because when we spoke earlier today, I said I'd get you some ideas, because we had nothing at the moment. Bob came through and we're good.

Kirk: Well, are they evergreen? Can we use them on a future show?

Chris: Absolutely.

Kirk: Okay.

Chris: Well, after you read it, you'll giggle and go, "Wow, this may be worth doing."

Kirk: Okay. I'll check it. Coming up on future...

Chris: There are-

Kirk: Yeah?

Chris: There are two pictures in there, so at least one of the pictures will make you laugh.

Kirk: All right. I'll look at it.

Coming up on next week's show, Andy Laird is here. Not only extraordinary broadcast engineer, but he can weight lift a ton. Is that odd? Is that weird? He's a champion.

Then a week after that, Ted Alexander, who probably can't weight lift a whole lot, but he's one of the best tech support guys that there ever has been.

So check us out. Thanks for watching, and be sure you tune in to the other programs that are on the GFQ network. There are some really cool things. Andrew Zarian has put together just an all-star cast, week after week, of programs that you want to watch. So keep it tuned here to the GFQ network. Check them out. Use your favorite pod-catcher to download the stuff and watch it at your convenience, if you can't watch it live.

Thanks to our sponsors, Lawo, ProSTREAM, and Axia for sponsoring our show this week. Thanks again to Chris Tobin and Bob Holowenko. We'll see you next week on This Week in Radio Tech. Bye-bye everybody.

Telos Alliance has led the audio industry’s innovation in Broadcast Audio, Digital Mixing & Mastering, Audio Processors & Compression, Broadcast Mixing Consoles, Audio Interfaces, AoIP & VoIP for over three decades. The Telos Alliance family of products include Telos® Systems, Omnia® Audio, Axia® Audio, Linear Acoustic®, 25-Seven® Systems, Minnetonka™ Audio and Jünger Audio. Covering all ranges of Audio Applications for Radio & Television from Telos Infinity IP Intercom Systems, Jünger Audio AIXpressor Audio Processor, Omnia 11 Radio Processors, Axia Networked Quasar Broadcast Mixing Consoles and Linear Acoustic AMS Audio Quality Loudness Monitoring and 25-Seven TVC-15 Watermark Analyzer & Monitor. Telos Alliance offers audio solutions for any and every Radio, Television, Live Events, Podcast & Live Streaming Studio With Telos Alliance “Broadcast Without Limits.”