My Mesh Network

[urgodfather]

So network capabilities is definitely a requirement in our home.... with me having my hands in so many things, I don't have the time to deal with the troubles of networking. At the same time, I don't own, so I cannot go pulling wire for a backbone. Because of this, I took what I had and expanded on it.

Over the last 6 years, I've used apple Airport Extremes. Some of you may cry and shout, but the truth is that they have simplified the process to an effortless setup and their wireless bridging just works.

Now that I'm able to get 400 x 20, it was time to do an upgrade. PS4 and IPTV, and multiple web and file servers, my network needed an upgrade. So, here's what I did.

Well before I jump right into it, there's some back story. When I established my account, the technician decommissioned every line in the house. Why? Because they think it will cause interference! What a load of BS! After fighting with them about this, the tech documented my account about the dispute and stated that if lines were reconnected that I would be subject to all costs related to any trouble ticket at my location and any neighbors if determined that my doing so was the cause. My response, "OK, you want to tell me how to hook up my house... so be it. Your job stops at the modem!"

Recently I had an issue with the ISP that took 12 visits over a period of 2 weeks to get them to identify and address my bandwidth being an average of 12 x 20 and merely peaking at 50 for a second. Turns out they had the main circuit oversubscribed (I tried to tell them but since I don't work for them, I must not know what I'm talking about). It only took a plant engineer to spend his day here to figure it out. Sadly for him, he spent the whole day while I was at work managing my own enterprise subscriber based network. When I got home, I explained everything to him, showed him the logs questioning why I was being routed from one city to another then back again, and then a completely unusual route to the mid country (MS) all to get routed back to ATL when ATL is a straight route from me. After showing them this, and the other iperf logs, I reiterated load balancing and routing policies. Sure enough they figured it out. Its a shame it took them so long as this issue affected the entire city!!!

OK so back on track, this new bandwidth and also the issues above contributed to my upgrade and configuration

My configuration is as follows.

Line to house >> box on house >> direct line to modem (end of ISP circuit)

Modem >> EdgeRouter >> PfSense on the side as transparent monitor (separate machine) and a Pi running PiHole for DNS and ad-blocking.
EdgeRouter >> MoCA adapter >> box on house >> separate MoCA compatible splitter >> rest of house
rest of house :
box on house >> MoCA adapter >> Airport Extreme in AP mode (Living room)
box on house >> MoCA adapter >> Airport TimeCapsule in AP mode (Den)
box on house >> MoCA splitter >> MoCA adapter >> Airport Extreme in AP mode (office)
box on house >> MoCA splitter >> MoCA adapter >> Airport Extreme in AP mode (master)

The other perk to me using the Airports is that each gives me an additional 3 ports to use for hardline connection.

In the den I have my Synology, PS3, PS4, Sony home theater, and my "entertainment box" all hardwired.

In the office, I have my main workhorse computer, my secondary node, my openhab controller, and my pi.

I have full signal for most of my cul de sac, my max ping internal is 18ms wheras my external is 24ms. I get full 400 x 20 hardwired and by WiFi but the other nice thing is my Airports support a "Guest" WiFi. Well this "guest" is now the everyday use wifi, and the primary is reserved for me as I work with some heavy files.

Everything works beautifully! I don't have to compete with friends, family, guests, misc. smart devices that are all on the "guest." My media is fully accessible as that "guest" wifi is a hidden VLAN used by Apple. So had to make the pinhole for it to get through.

I have DPI running, Snort and pihole. I also can enjoy 4K locally on every TV without a hiccup in the network.

Since my setup, the ISP has followed up 3 times to "check on things." The first time was the same tech that marked up my account during the initial install. When he called his supervisor and the sent a picture to the Engineer, they were told the same thing I said, the line to the modem is dedicated and separate. Your responsibilities stop there.

Along with this, because of the beauty of my setup, there are no additional holes, wires, etc. and I can take it down in a flash without a trace. Pretty sweet!

[eccerr0r]

Re: My Mesh Network

Yeah I had the same thoughts, plus I shudder when I use wifi, if your neighbors are close, remember you're sharing the bandwidth with your neighbors too.

On the other hand cat5 might be cheap but burying it behind walls, etc., is expensive... that is, if your significant other protests to seeing it...

[Curious.George]

Re: My Mesh Network

Yeah I had the same thoughts, plus I shudder when I use wifi, if your neighbors are close, remember you're sharing the bandwidth with your neighbors too.

And anything else that wants to pollute the spectrum -- deliberately or accidentally. Imagine that movie stream being interrupted for a couple of seconds...

On the other hand cat5 might be cheap but burying it behind walls, etc., is expensive... that is, if your significant other protests to seeing it...

So, you're aggregate bandwidth is considerably higher -- each drop giving you an incremental multiplier.

And, with real copper running to each drop, you can also distribute power to those devices without having wall-warts scattered all over the place. Additionally, as you would then be powering from a central location, you can backup that power source and effectively backup all of those devices at the same time!

[urgodfather]

Re: My Mesh Network

Being that I work in IT, a lot of the stuff I have accumulated over the years.

1. The Apple devices are definitely the most expensive components. Whats nice about their equipment, even though they have now been discontinued by Apple, is that they "talk" to one another, similar to the MoCA devices. Since all of them are all in AP mode, I am channel steering to target desired channels. I rarely have an issue.
2. The MoCA adapters were the most recent component. I bought them for 20 a piece. The devices I'm using aren't even technically available on the market and you can't even get any literature on the. Trust me, I called Zebra, Motorola, Symbol, Arris... (all the same company btw) none of them have any documentation on them. The only thing I could find is a small reference to this being an add-on component to a modem. That's it. The splitters are dirt cheap, and 10 extra ft of dual shielded coax was easy to come by. I already had the compression tool and fittings. There are also low power devices that run off of 5V. I have USB-A to USB-A wires connected to the Airport Extremes powering them.
3. The Pfsense box is a small machine that was decommissioned. A fully functional i7 vPro with 120 Gb SSD and dual nics that only had a bad RAM stick in it is definitely worth not disposing of.
4. The pi again also something I had laying around. I actually have several. Not to mention various other development boards, doo dads and what nots.
5. The edgerouter is dirt cheap. I think they sell on Amazon for 40 bucks now. Still does what I need. And if I get tired of it, I can always change it out for the Pfsense playing its role too. Or I could pop in one of several enterprise grade firewalls I have laying around.

*If I were rebuild it, I would probably use UniFi AP's and USG along with setting up my own cloud controller. Their equipment is cheap and powerful. I would still keep the MoCA and the Pfsense and PiHole setups.
**If not using UniFi gear, then I would probably buy some MoCA adapters with integrated WiFi. Again, I would still channel and RSSI steer it.

Expanding on my home network, I have actually added more since my initial post. Now I have integrated Zigbee and Zwave. I have some smart thermostats (not a Nest or EcoBee) and a couple smart plugs, along with a controller running the automation.

The fundamental thing to remember is that networking isn't just confined to WiFi and Ethernet. There is a lot of different technologies out there, all of which are application specific networking. Modern cars for example have multi-layer networks that allow the components to communicate. Some on different levels than others, but still.

In regards to the comment about CAT5 wire being cheap, you have to consider the additional fact that it takes time to plan, route, fish and tidy up CAT wire. Not to mention the fact that its a permanent change to a house that I don't own. The owner wouldn't want me to do something so invasive. Everything I have integrated can be pulled out in no time. No major modifications other than the 2nd coax wire going into the garage for the internal coax network and the mounting of the thermostats.

I also have the majority of stuff set up on UPS's. Some are 1500's, some 1000's and a couple of 750's scattered about in the house. If power goes down, I can sustain the whole network for up to 4 hours (provided I don't use unnecessary draw like my major workhorse machine). Afterwards things would gradually start dying off. Because the way I have things layed out, I could still have wireless throughout the house for around 6 hours and if push came to shove, I could always go direct connect for a couple more hours. My largest batteries are at the major locations.. backbone equipment and den.

[Curious.George]

Re: My Mesh Network

Expanding on my home network, I have actually added more since my initial post. Now I have integrated Zigbee and Zwave. I have some smart thermostats (not a Nest or EcoBee) and a couple smart plugs, along with a controller running the automation.

And it's all still wireless. When you move into that crowded apartment building (or subdivision) and drag all of the kit you pull from this place along and discover that "something" is limiting your throughput to acoustical modem speeds, what alternative will you have?

I have a buddy who bought a wireless TV, excited because it let him "cut the cord" and locate it anywhere in the house. Until he started getting pixelated video as the bandwidth available fell.

The fundamental thing to remember is that networking isn't just confined to WiFi and Ethernet. There is a lot of different technologies out there, all of which are application specific networking. Modern cars for example have multi-layer networks that allow the components to communicate. Some on different levels than others, but still.

And modern cars run copper to their scores of network nodes! In the quantities that they purchase, they could put a wireless technology at each node (after all, a car isn't even as big as a bedroom!) and skip the wire and the cost of running it! Ah, but they need to run power to those nodes, too, so...

In regards to the comment about CAT5 wire being cheap, you have to consider the additional fact that it takes time to plan, route, fish and tidy up CAT wire. Not to mention the fact that its a permanent change to a house that I don't own. The owner wouldn't want me to do something so invasive. Everything I have integrated can be pulled out in no time. No major modifications other than the 2nd coax wire going into the garage for the internal coax network and the mounting of the thermostats.

So, I guess the landlord DOESN'T have a problem with the WIRING ADDITIONS you've made?

Every such "upgrade" I performed on rental properties was gladly accepted by the landlord -- it cost him nothing and he's got some extra "selling point"!

Running cable is a PITA. That's how wireless can appeal to The Masses (i.e., expect MORE stuff in the ether with each passing day). But, with each cost/convenience comes an advantage/liability!

I've just shy of 5000 ft of CAT5, here (modest residential home). Without a basement or attic to snake cable through, it was very costly (tedious) to install!

But, I don't worry about someone mounting a DoS attack on those devices. What does your doorbell camera do when someone (thief/burglar) points a signal jammer at it? ("That's illegal!" "Yeah, so is Breaking and Entering!") Or, a genuine attack on their security protocols! Like when he eavesdrops on the comms -- OR, uses the "remote" capability to hack it from the comfort of his car, across the street??

And, I don't have a sh*tload of little wall warts scattered around powering all of those PD's (cameras, phones, STB's, thermostat, etc.)! Because of this, I can hide lots of kit without leaving visible "blemishes" to belie their presence (e.g., my APs are located IN the ceiling or behind walls).

I also have the majority of stuff set up on UPS's. Some are 1500's, some 1000's and a couple of 750's scattered about in the house. If power goes down, I can sustain the whole network for up to 4 hours (provided I don't use unnecessary draw like my major workhorse machine). Afterwards things would gradually start dying off. Because the way I have things layed out, I could still have wireless throughout the house for around 6 hours and if push came to shove, I could always go direct connect for a couple more hours. My largest batteries are at the major locations.. backbone equipment and den.

You're backing up big pieces of kit. Not phones, cameras, etc. I have 7 UPSs in the office, one on this "visible" machine, one on the TV and two in my bedroom. These are for "big pieces of kit" (e.g., each set of workstations in the office has 4 monitors that need to be powered in addition to the "CPUs"). I can get varying degrees of uptime from each UPs, depending on the loads being powered. E.g., this machine -- plus the wireless ISP transceiver/modem -- will run, "as is" (i.e., IN USE, connected to The Internet) for just over two hours.

But, that doesn't count the UPS that powers the kit that's distributed around the house (phones, cameras, microphones, speakers, etc.). That UPS is located in an electronics closet alongside the main PoE switch. Powering every conceivable device (2000W), it wouldn't stay up for more than 15 minutes! (I currently have 700WHr of battery available, there)

But, the load is smart and knows how to shrink when running on battery!

"Sorry, you can't pipe your favorite tunes into the garage, now -- we don't have excess battery capacity to "waste" on things like that (more important to keep the security cameras, fall monitors and phones running!)"

If everything was powered with wall warts, you'd have to place a bunch of UPSs around the house to backup each of those. And, any "battery reserve" available in the UPS sitting by the living room phone couldn't be exploited to power the security camera in the back yard.

This may seem "frivolous", today, but when your home starts accumulating (ZigBee, Xwave, BT, etc.) IoT devices, you'll quickly learn what eyesores they will be AND their vulnerabilities (to interference and power outages). You'll find the same convenience that drew you to that kit will cause you to abandon it "when you get serious".

[urgodfather]

Re: My Mesh Network

You realize the infrastructure in my setup is still copper, right?

Secondly, who in their right mind would install a smart door lock? Some things are best kept analog.

You keep mentioning wall warts but I don't have warts all over the place.

In regards to radio interference, that would only happen if the frequencies even collided. Tell ya what, I'll take the time to build a spectrum analyzer. Then we will see how noisy it is with my environment.

Also, none of this was some kit.

Oh, my modem stays unaffected BTW. Just saying... I get 20% more performance than I'm rated for.

I guess I shouldn't mention that my surround sound in my den is S-air. LOL

[Curious.George]

Re: My Mesh Network

Secondly, who in their right mind would install a smart door lock? Some things are best kept analog.

Why? How would you let your neighbors into the house to water your plants in the event of your (unexpected) absence?

There's nothing wrong with "smart". There's PLENTY wrong with things CLAIMING to be smart that really aren't!

You keep mentioning wall warts but I don't have warts all over the place.

Because you only have traditional/legacy kit! You don't have VoIP phones, network audio (throughout the house), smart window coverings, a means of opening and closing the garage door WITHOUT a keyfob, IP video cameras, smart HVAC controls, etc. Each of those "things" needing mains power (or, worse!, batteries) to operate.

But, you WILL -- sooner than you realize! And, you'll find your past wireless experience won't scale to meet those needs. But, your mindset will be stuck on avoiding the wires -- until you find things "just not working" (well).

In regards to radio interference, that would only happen if the frequencies even collided. Tell ya what, I'll take the time to build a spectrum analyzer. Then we will see how noisy it is with my environment.

As more of your neighbors adopt the same "its easy -- just plug it into power" approach, the spectrum gets increasingly congested.

And, an adversary will have no problem deliberately pushing The Right Noise at you if he realizes (knows!) he can use that to circumvent some aspect of your "system" (e.g., all of those easy to install WIRELESS reed switches that monitor for door opens or glass breakage). Note that he can go to the same STORE from which you purchased your kit and buy identical devices and reverse engineer them, at his leisure.

Of course, no one is likely to do that JUST to target YOU. But, someone will have done it and shared it with others who will need no technical knowledge to apply it to folks who've adopted that particular technology. The hadware equivalent of "script kiddies".

I guess I shouldn't mention that my surround sound in my den is S-air. LOL

My sound follows me around the house -- turning off in the room I've just left and picking up, uninterrupted, in the room I'm entering (unless, of course, it would be competing with some other audio program currently being routed to that room -- which might include an incoming phone call or visitor at the front door!)

[When I finish the design of the video client, the same sort of thing will be true of any movies/video that I'm watching]

To date, IoT devices have been toys with very limited application and virtually NO integration. Little more than X-10-like remotes that you can control (from a server located at Amazon.com or Google.com). That will change in the reasonably near future as these devices become more pervasive. And, each instance will require comms AND power.