Remote pump control ideas.

[Dannyx]

Re: Remote pump control ideas.

Sorry, I've lost track of what the desired goal of all of this is (and I'm not keen on reading 80 posts to sort it out!).

Yeah, it's my fault. I said I wouldn't shoehorn another project in here which has nothing to do with the original one, so let's fold back.

It all started with trying to control a three-phase pump based on inputs from a mechanical switch which is located kms away from the pump. What I have to work with is a GPON and internet connection in each shack, from a hardware perspective...I have no idea what the boys over at our ISP department actually DO with these, but it's their job to make the two boards "see" each other, while I on the other hand, have to make them "talk" TO each other....which I did successfully on the bench (even with some rudimentary checks to make sure each run/stop command was actually received, as previously discussed) though no programmer would hurry to praise me for the frankenstein I created

[Curious.George]

Re: Remote pump control ideas.

BTW, any suggestions for small form-factor boards with wired Ethernet ? The Nucleo 144 is WAY too big physically for building small "clients" - it's a waste of both space and hardware to just read 1-2 pins on it. I imagine something like an ESP8266 but with wired Ethernet...or a more hacky way: an arduino Pro Mini or Nano with an ethernet shield patched onto it with jumper leads, which is what I currently have on my workbench to test all these projects

(sigh)

Sorry, I've lost track of what the desired goal of all of this is (and I'm not keen on reading 80 posts to sort it out!).

Think about what you are trying to do "in the abstract".

Think about the constraints (physical, financial, temporal) that you have to address.

Think about the resources (money, time, expertise) that you have available.

Remember, wired ethernet will require a switch (or switches), somewhere. Accommodating that switch can suggest other communication topologies (e.g., it's a physical star but a logical bus; perhaps consider a physical star AND logical star? i.e., have everything plug into a "master")

Also remember that you can implement networks with a variety of hardware technologies and protocols. E.g., you could implement an RS485 network and layer a token-passing protocol atop it (to eliminate the concept of a "master"). Or, a TDM access scheme driven by the unique ID of each node. Or...

You may find that your data rates are low enough that slower, less complex hardware can address your needs. Or, may find the wire distances involved make some solutions "challenging".

The appeal that "ethernet" has is that it is a standard base for many protocols and services "out-of-the-box". E.g., you can layer a web service atop a TCP/IP stack in a jiffy. Trying to offer the same service on some other protocol/hardware means a fair bit of work (esp if you want to use standard browers)

You may not need that flexibility. But, be sure that you WON'T at some future date! It's really hard to tell an employer/client that you've got to scrap their existing hardware AND software "just" to add one of these sorts of capabilities.

[stj]

Re: Remote pump control ideas.

ARDUINO is not hardware, it's the software platform which is based on the GCC C compiler with lots of libraries to make it simple to manipulate the hardware.

[Dannyx]

Re: Remote pump control ideas.

BTW, any suggestions for small form-factor boards with wired Ethernet ? The Nucleo 144 is WAY too big physically for building small "clients" - it's a waste of both space and hardware to just read 1-2 pins on it. I imagine something like an ESP8266 but with wired Ethernet...or a more hacky way: an arduino Pro Mini or Nano with an ethernet shield patched onto it with jumper leads, which is what I currently have on my workbench to test all these projects

[Dannyx]

Re: Remote pump control ideas.

I have no idea what their library supports. My (uninformed) opinion of these devices is that they aren't really intended for "serious" work.

Well, no, they're not - the Arduinos are for people getting started with this sort of thing. You CAN get into more complex stuff with them, but they're not industrial-grade. They're useful for small projects like these which do very little.

[Curious.George]

Re: Remote pump control ideas.

So I thought. Makes perfect sense and is completely logical, but unfortunately this is slightly limited in the case of the Wiz W5x00 series the arduino library works with, that's why I thought I'd look into more advanced ways of doing, but not too over the top - I'm not planning to become a master coder overnight. Just enough to save my life and design simple projects like these to keep my boss happy while at the same time learning from mistakes

I have no idea what their library supports. My (uninformed) opinion of these devices is that they aren't really intended for "serious" work.

I had a look at some tutorial vids on the STM32 series....it's actually overwhelming. The difficulty curve is very steep. So they use their own IDE - "Eclipse" (much like the Arduino has its own), but since this is not aimed at beginners, what I noticed is that it likes to dump everything it uses in the code window, which the Arduino IDE doesn't do, since it spares the headaches of newbies. For instance, it includes the stuff which is normally hidden behind the libraries, like the parameters for UART and other peripherals - it's all there in the code window, so you have to really know what you're doing, what to modify and what not to modify. I was able to follow along and understand the "while(1)" loop is the equivalent of void loop() in Arduino, where I'd put my main code. Not sure if it'd be as easy as copying and pasting though - the pin numbering wouldn't match for one thing and even after that, I'm not sure all the functions are compatible...

The part I'm mostly interested in, despite being too early, is how you deploy ethernet on one of those. Even if I include the STM32 boards in the Arduino IDE (which CAN be done apparently, for beginners who wish to play with more powerful boards while still maintaining the simple IDE), the ethernet library for the Arduino is not going to work here because it's not using the same chipset, so I have to use the Eclipse for this.....sucks that time is so scarce and everybody wants stuff done yesterday, but I guess that's the price I pay for embarking on such a journey

"Eclipse" is pretty much the de facto IDE for most "serious" software development work.

If you plan on using ethernet to talk between devices/nodes, you have two ways to go.

If you want to handle a lot of the details of the communication protocol yourself, you don't need to rely on having a full network stack. You can, instead, build a protocol on UDP which can be implemented "cheaply".

But, UDP leaves lots of details up to you to implement (message sequencing, retries, connection handling, etc.). It's essentially just pushing bytes down the wire and hoping that your devices manage to capture them as they pass. If a message gets corrupted (or missed!), then YOU have to implement some protocol for resending it.

[This might not be an issue if you have lightly loaded devices on the same network segment with little other network traffic; but, stick a router in there or other protocol traffic on the wire and its harder to guarantee correct operation -- that's why TCP was invented!]

If you want a "virtual connection" between a program on one device (a client) and another program on another device (a server), then TCP makes your life REALLY easy; you talk directly to the other party and the network stack handles all the details of error correction, timeouts, packet reassembly, retransmissions, routing, etc.

A (developer) friendlier approach might be to use something like Inferno. But, you'd need to find a port for your chipset -- likely not available for "oddball" devices. (I think an rPi port may be available)

[Dannyx]

Re: Remote pump control ideas.

A "server" is typically a single device. It can handle multiple clients, concurrently, using a number of different implementation mechanisms.

So I thought. Makes perfect sense and is completely logical, but unfortunately this is slightly limited in the case of the Wiz W5x00 series the arduino library works with, that's why I thought I'd look into more advanced ways of doing, but not too over the top - I'm not planning to become a master coder overnight. Just enough to save my life and design simple projects like these to keep my boss happy while at the same time learning from mistakes

I had a look at some tutorial vids on the STM32 series....it's actually overwhelming. The difficulty curve is very steep. So they use their own IDE - "Eclipse" (much like the Arduino has its own), but since this is not aimed at beginners, what I noticed is that it likes to dump everything it uses in the code window, which the Arduino IDE doesn't do, since it spares the headaches of newbies. For instance, it includes the stuff which is normally hidden behind the libraries, like the parameters for UART and other peripherals - it's all there in the code window, so you have to really know what you're doing, what to modify and what not to modify. I was able to follow along and understand the "while(1)" loop is the equivalent of void loop() in Arduino, where I'd put my main code. Not sure if it'd be as easy as copying and pasting though - the pin numbering wouldn't match for one thing and even after that, I'm not sure all the functions are compatible...

The part I'm mostly interested in, despite being too early, is how you deploy ethernet on one of those. Even if I include the STM32 boards in the Arduino IDE (which CAN be done apparently, for beginners who wish to play with more powerful boards while still maintaining the simple IDE), the ethernet library for the Arduino is not going to work here because it's not using the same chipset, so I have to use the Eclipse for this.....sucks that time is so scarce and everybody wants stuff done yesterday, but I guess that's the price I pay for embarking on such a journey

[Curious.George]

Re: Remote pump control ideas.

Sort-of....This was the idea that came to my mind after what STJ suggested (EDIT: OK, no, he didn't actually suggest THIS, but I worked around it): have multiple servers which listen for the single incoming client - sounds stupid, but strictly from the perspective of the hardware I'm working with at the moment, it's the only workaround that I, the average Joe, sees Hence why I love discussing stuff like this and learning how more advanced chaps do it.

A "server" is typically a single device. It can handle multiple clients, concurrently, using a number of different implementation mechanisms.

You would, thus, have a single node that exports a particular "service". Clients on each of the other nodes would then connect to that service.

Note that -- with ethernet protocols -- you always want to be a good "net citizen" as it's almost certain that, sooner or later, you will have to coexist with other protocols/services running on that network. So, you want to "do things right" instead of ad hoc.

[stj]

Re: Remote pump control ideas.

yes, if they dont clash on the network.
so dont poll the slaves multiple times a second and it should work fine

[Dannyx]

Re: Remote pump control ideas.

it would help if you call them master and slaves.
master commands, slaves respond when spoken to.

Makes sense, but can there be multiple masters ? I'm sure of it...

[stj]

Re: Remote pump control ideas.

it would help if you call them master and slaves.
master commands, slaves respond when spoken to.

[Dannyx]

Re: Remote pump control ideas.

Does each client "discover" the identity of the (single?) server?

I WAS planning on doing it this way, though I haven't actually TRIED it on the bench, due to a lack of both inspiration and hardware. Reading about it though, like I said, revealed some issues. On paper, it seemed like it'd be totally doable, as described - different IPs and junk - and makes perfect sense, but from a practical standpoint, apparently there are some limitations, like I said...at least that's what I've read when others tried the same thing.

Does the server discover the identities of its multiple clients?

Sort-of....This was the idea that came to my mind after what STJ suggested (EDIT: OK, no, he didn't actually suggest THIS, but I worked around it): have multiple servers which listen for the single incoming client - sounds stupid, but strictly from the perspective of the hardware I'm working with at the moment, it's the only workaround that I, the average Joe, sees Hence why I love discussing stuff like this and learning how more advanced chaps do it.

This is also why I was curious about how other boards, other than Atmels, handle Ethernet, because frankly I haven't left that realm yet, so I'm still in the stone-age so to speak...

[Curious.George]

Re: Remote pump control ideas.

Yes, but from a code perspective, I don't see a way for the "server" board to actually initiate the connection to each client. The commands here don't make it seem so at least. Server.begin() just makes it "listen" for clients, but IT can't actually send anything out until a client willingly connects and sends something first....I think...that's how I did it, but like I said, could be way better...the more you learn

What you describe definitely sounds a lot like Modbus which I read about.

You have to choose how your system configuration is managed.

Do you "tell" someone that there are N clients and they exist at the following IP addresses?

Is each client told of the identity of its server? (what if there are multiple servers on the same internet??) Or, does each client "discover" the identity of the (single?) server?

Is the server told of the identities of each of its clients? (what if there are clients that will NOT be serviced by that server?) Or, does the server discover the identities of its multiple clients?

This decision has consequences. What if a client goes away? What if a new client is added? Are IPs statically assigned or dynamically allocated? Are certain IPs bound to certain MACs? etc.

Think about how you want the user/administrator to deal with your system before thinking about how to implement it.

[Curious.George]

Re: Remote pump control ideas.

I just thought of something I thought would be trivial but could end up being tougher than I thought: connecting multiple clients to a single server....not too sure yet what hardware I'll be using and how this affects the difficulty of this task.

For the pump project, this is easy: one client - one server, but here's there's several "ears" which have to report to the server which has to differentiate between them (separate warning light for each of them) and I don't think they can all hog the same port (23 in the case of the pump project)...I think I have to create multiple server objects on the server board and have each client connect to its own server. I think we're talking about multi-threading capabilities here which requires more advanced hardware...

You don't understand how protocol stacks work.

A TCP connection is defined by a 5-tuple:

(protocol, source IP, source port, target IP, target port)

A server can manage any number (depends on available resources) of connections.

For example (neglect "protocol" as we assume they will all be using the same protocol, in your applicaation):

(clientA IP, clientA port, server IP, server port)
(clientB IP, clientB port, server IP, server port)
(clientC IP, clientC port, server IP, server port)
(clientD IP, clientD port, server IP, server port)
(clientE IP, clientE port, server IP, server port)

Yes, the "server IP" and "server port" are constant, but, these are all different -- because each "client IP" is unique from all others. So, the server's stack knows that the first tuple is used to talk with clientA while the fourth is used to talk with clientD.

[Dannyx]

Re: Remote pump control ideas.

the "master" is the monitoring station with all the lights & shit,
the stations would be considered clients/slaves because they answer requests rather than making them.

Yes, but from a code perspective, I don't see a way for the "server" board to actually initiate the connection to each client. The commands here don't make it seem so at least. Server.begin() just makes it "listen" for clients, but IT can't actually send anything out until a client willingly connects and sends something first....I think...that's how I did it, but like I said, could be way better...the more you learn

What you describe definitely sounds a lot like Modbus which I read about.

[stj]

Re: Remote pump control ideas.

the "master" is the monitoring station with all the lights & shit,
the stations would be considered clients/slaves because they answer requests rather than making them.

and this method can also be used with multi-point serial like rs485
and wifi/bluetooth/descrete-radio modules - but i dont recommend those for the reason i already gave.

[Dannyx]

Re: Remote pump control ideas.

stop reading what your reading.

your stations listen and have a unique i.d.
your base poll's each station once per second and request the status.
if a station does not respond then it flags an error.
if a station responds with a status of "button pressed" or "fire sensor" or whatever then it sets the lights or display and does the sounder or whatever
simple

the stations themselves just constantly listen on the same port for a packet that contains their id byte.
if they dont match then the buffer is cleared, if they do match then they reply.
a router will multicast across all ports if you use the right packet.

That's a different approach indeed. The way I had it set up was based on the pump idea where a client connects to a server and that's it.

Sounds like what you're suggesting is to have multiple servers instead and a single client which initiates the connection to each one by one....good idea in fact, though it requires some massive work at this stage.

So the servers just sit there waiting for incoming clients and when the single client hooks up (the "base" where the staff is), I'd have to have each server ("Station") report its ID and last status, then the client would have to disconnect, move to the next one and repeat. This means I'd have to move my "client.connect()" function outside the "setup()" section, since I'd have to step through the IPs rather than get stuck with a single one...just some random thoughts as I think about this. Nice idea, actually....will certainly keep me busy It also proves that I still can't think like a programmer and come up with solutions to problems - I take the "hard" approach...

[stj]

Re: Remote pump control ideas.

stop reading what your reading.

your stations listen and have a unique i.d.
your base poll's each station once per second and request the status.
if a station does not respond then it flags an error.
if a station responds with a status of "button pressed" or "fire sensor" or whatever then it sets the lights or display and does the sounder or whatever
simple

the stations themselves just constantly listen on the same port for a packet that contains their id byte.
if they dont match then the buffer is cleared, if they do match then they reply.
a router will multicast across all ports if you use the right packet.

[Dannyx]

Re: Remote pump control ideas.

I too thought it would be simple conceptually, but as I read online, there are some snags one might hit with this multiple-clients-to-one-server business, limited strictly to the Arduino world: you can't use the same port for both clients (not without extensive mods to the ethernet library) and you can't have more than 4 clients connected at once (limitation of the W5100 shield). In this case, I'm not sure how many rooms there are which need a client in them and whether this would be a problem.

I started looking into more advanced and exotic ways of doing this and found THIS. Trouble is I have never messed with a Raspberry Pi before and can't experiment anything at the moment because I don't have one.

Then there's also that Mbed for ARM boards....wonder how those handle TCP/IP and if it's wildly different than the library for the Arduino shields or still shares some concepts. Like I said, all these choices make a novice's brain hurt when having to choose the best one

[stj]

Re: Remote pump control ideas.

the clients just have to transmit 2 bytes.
a unique id, and a status byte.