10 hours lost because stupid laptop wouldn't boot! GRRR!

[eccerr0r]

Re: 10 hours lost because stupid laptop wouldn't boot! GRRR!

The problem is that you should know what to expect. Some people know what to expect and you have no right to complain about their practices. And if you don't know what to expect, you get your machine blown up, plain and simple. This is especially true if you're not watching it, whether it's on your own personal machine running a long term task or on a server that you don't have access to the console.

I run Gentoo and see these potential problems every time I do software updates on multiple machines. Logfile fillups have never been a problem for me building hundreds of packages over the years, I *know* when disk should be used and stop it before it becomes a problem, if it becomes a problem at all. Even the system logs kept for long term analysis have never an issue.

400 days uptime is a useless "brag". When there's a library update, chances are, you need to reboot, else you will not get all instances of the software replaced in memory. Windows forced reboots for a reason, Un*x users get complacent and think "OH, NO REBOOT NEEDED!" and get 0wn3d when someone exploits an old copy of a library still in memory, like /sbin/init's copy of libc and tons of other daemons that never gets restarted because, oh no, it would drop your availability because you really needed to restart almost every service because they depend on each other ... oh too bad, so sad.

Looks like you should take another look at your practices instead of bitching at someone else for differing with your own problems, and especially creating nonsense to inflate your oh so big ego.

[petehall347]

Re: 10 hours lost because stupid laptop wouldn't boot! GRRR!

some tunes to listen to .. sorry about the crackles .. dedicating it to log files .. last one hang on just sorting tape out .. https://www.youtube.com/watch?v=k_LOLH6nMB0

[Curious.George]

Re: 10 hours lost because stupid laptop wouldn't boot! GRRR!

The problem is that you should know what to expect. Some people know what to expect and you have no right to complain about their practices. And if you don't know what to expect, you get your machine blown up, plain and simple. This is especially true if you're not watching it, whether it's on your own personal machine running a long term task or on a server that you don't have access to the console.

No, you shouldn't "know what to expect" when building sources that other people have written. How WOULD you know? Build them, see what happens, then go back in time with that knowledge and "know what to expect"? Nor should you have to "watch over it" -- just in case...

Do you know how much free disk space your system needs in order to rebuild a kernel? What about the entire distribution? Do you know how long it is likely to take? How are you planning on finding out??

Have you tried to hammer away at the ftp service on your box to see when/if it will crash? Ditto HTTPd? Or, a ping flood? Finger war? Do you know what happens if the drive that serves the /usr partition "goes flakey"? How long will init(8) sit there trying to mount it if the drive is ALMOST working? Will you be able to interact with the system while its struggling in this way? (it hasn't yet left single-user)

Instead, you should design/configure your system so that "the unexpected" doesn't bring it to its knees.

I run Gentoo and see these potential problems every time I do software updates on multiple machines. Logfile fillups have never been a problem for me building hundreds of packages over the years, I *know* when disk should be used and stop it before it becomes a problem, if it becomes a problem at all. Even the system logs kept for long term analysis have never an issue.

Building hundreds of packages OVER THE YEARS is far different from building thousands of packages OVER THE DAYS. There is a considerably higher processor utilization factor that directly translates to the number of log entries generated per unit time. You're talking about millions of commands being executed, right after each other, in a short time period.

Had you been sitting at the workstation, would you have noticed that freespace was decreasing at an alarming rate? If /var had NOT been a separate partition (that you can easily and directly inspect), would you have known where to go looking for the "consumption" -- before you ran out of space? If other users had been connected to the machine at the time, how would you have explained their loss of service to them?

400 days uptime is a useless "brag". When there's a library update, chances are, you need to reboot, else you will not get all instances of the software replaced in memory. Windows forced reboots for a reason, Un*x users get complacent and think "OH, NO REBOOT NEEDED!" and get 0wn3d when someone exploits an old copy of a library still in memory, like /sbin/init's copy of libc and tons of other daemons that never gets restarted because, oh no, it would drop your availability because you really needed to restart almost every service because they depend on each other ... oh too bad, so sad.

Linux may be designed so that every service depends on each other but that's not true of the "other" Eunices. seems like a pretty foolish way to design a system, no?

And, we're not talking about a windows machine now, are we? You can replace libraries and binaries of ANYTHING (short of the kernel) without a reboot. Rebooting is for the folks who don't know "what's where" in their binaries.

A competent sysadm can replace an entire system with minimal inconvenience to its users by judicious use of mv's and symlinks. An application IN use can't be altered FOR ITS CURRENT USER. But, any invocation of that application by another user can be directed around the application in question:

# mv /usr/bin/app /usr/bin/oldapp
# mv /usr/bin/newapp /usr/bin/app

At this point, any instances of app that are currently running remain the "old" version of the application. Any new instances of "app" that are started will invoke the "new" version -- even if invoked by a user who happens to have an instance of the old application currently running!

But, you knew that, right? :>

Likewise for libraries and shared libraries.

Services that tend not to want to be stopped often support a signal to allow for their configurations to be reloaded. I can swap all the (name,IP) bindings on my network and have the new bindings take effect in the time between keystrokes. Take a printer out of service -- yet let it finish up anything that it happens to be printing, at the time. Change the display manager -- or its configuration -- on a whim.

And, services dispatched by inetd(8) can be updated simply by replacing (or renaming) the original application.

You can even replace the kernel while the system is running -- but the new binary won't be loaded until a genuine reboot (15 seconds to shutdown "-now" followed by another 15 to reboot):

# mv /kernel /okernel
# mv /newkernel /kernel
# reboot

Looks like you should take another look at your practices

Why? Mine work absolutely fine! But, are you sure YOURS do?

Sit down at your workstation (and don't get up cuz that will magically transform it into a SERVER -- make sure you've emptied your bladder cuz you're going to be there for a while!)

Now, assume whatever privilege level is required for your particular OS. Move to the top level directory of the source tree for your SYSTEM (not just your kernel). Type "make world" -- or its equivalent. All should go well, right? Cuz you already KNOW what to expect -- because "you should know what to expect"! Let us know how that goes for you...

instead of bitching at someone else for differing with your own problems, and especially creating nonsense to inflate your oh so big ego.

You might want to go back and reread my posts, here, and decide if you want to double down on that accusation. In particular, you'll note that I HAVE NO PROBLEMS with my configuration. Years of administering systems have taught me many of the ways you can get shot in the foot -- so, I've changed my "best practices" to reduce the possibility of those sorts of issues eating my lunch. If you want to remain ignorant, go ahead and get shot! Then, YOU can have the "experience" to share!

(sigh) Sounds like I've seriously wounded YOURS! But, then again, if your only use for a "computer" is to play games and visit on-line forums, I guess it would be relatively easy to bruise... or, do you have some magical machine that you NEVER have to wait for? I guess all those two-layer boards don't require much effort to layout (or design).

Or, is it all just "The red eight goes on the black nine..."?

[Curious.George]

Re: 10 hours lost because stupid laptop wouldn't boot! GRRR!

some tunes to listen to .. sorry about the crackles .. dedicating it to log files .. last one hang on just sorting tape out .. https://www.youtube.com/watch?v=k_LOLH6nMB0

Probably only makes sense to folks of a certain age/experience, but worth inserting, here:

The Twelve Days of Multics
(sung to the tune of The Twelve Days of Christmas)

On the twelfth day of MULTICS
My true love sent to me:
Twelve analysts twitching,
Eleven options switching,
Ten stacks a-popping,
Nine aids debugging,
Eight tapes attaching,
Seven DIM's a-dimming,
Six ITS a-pointing,
Five protection rings,
Four calling sequences,
Three link pairs,
Two system dumps,
And a file tree hierarchy!

[petehall347]

Re: 10 hours lost because stupid laptop wouldn't boot! GRRR!

https://www.youtube.com/watch?v=IF_BVhHRT54

[eccerr0r]

Re: 10 hours lost because stupid laptop wouldn't boot! GRRR!

No, you shouldn't "know what to expect" when building sources that other people have written. How WOULD you know? Build them, see what happens, then go back in time with that knowledge and "know what to expect"? Nor should you have to "watch over it" -- just in case...

Do you know how much free disk space your system needs in order to rebuild a kernel? What about the entire distribution? Do you know how long it is likely to take? How are you planning on finding out??

The key is to PLAN AHEAD. Isn't it obvious that you don't run something that takes 128GB with constant rewrites on your little class 10 32GB MicroSD card that has been overspecced by the manufacturer, and never tested it before that critical operation that must succeed?
And cry that it takes down the whole system with it when it fails?
Of course not. Plan ahead, plan accordingly.

Have you tried to hammer away at the ftp service on your box to see when/if it will crash? Ditto HTTPd? Or, a ping flood? Finger war? Do you know what happens if the drive that serves the /usr partition "goes flakey"? How long will init(8) sit there trying to mount it if the drive is ALMOST working? Will you be able to interact with the system while its struggling in this way? (it hasn't yet left single-user)

Seems like all you have is a hammer so everything looks like a nail. I see that's indeed what you've been doing for every post you write.

Instead, you should design/configure your system so that "the unexpected" doesn't bring it to its knees.

Of course. No need to restate the obvious. You can even predict some things with enough experience. Plan ahead.

Building hundreds of packages OVER THE YEARS is far different from building thousands of packages OVER THE DAYS. There is a considerably higher processor utilization factor that directly translates to the number of log entries generated per unit time. You're talking about millions of commands being executed, right after each other, in a short time period.

yes of course, hundreds of UNIQUE packages. I don't exaggerate and hence under report numbers to make it seem like less a deal than it really is, but yes a thousand over a few days is something that has happened over several occasions - though the goal is to not have that happen - it means a neglected system. Did you neglect yours for the multiple thousands of packages, of which many may have been exploited during that time until the batch update? My machines that needed 1000-package updates were powergapped.

Had you been sitting at the workstation, would you have noticed that freespace was decreasing at an alarming rate? If /var had NOT been a separate partition (that you can easily and directly inspect), would you have known where to go looking for the "consumption" -- before you ran out of space? If other users had been connected to the machine at the time, how would you have explained their loss of service to them?

Yes. Or use a program to do it... Or you plan for it as you write 2000 words when only 3 are needed.

Linux may be designed so that every service depends on each other but that's not true of the "other" Eunices. seems like a pretty foolish way to design a system, no?

And, we're not talking about a windows machine now, are we? You can replace libraries and binaries of ANYTHING (short of the kernel) without a reboot. Rebooting is for the folks who don't know "what's where" in their binaries.

Oh great you can go change things in TWO places to make sure the both stay in sync, so much for letting the computer do that work. And go reread again, there are things you simply cannot replace a file, directory, etc., and expect everything to work shiny and new. Someone has to suffer downtime once in a while, just need to plan for it.

A competent sysadm can replace an entire system with minimal inconvenience to its users by judicious use of mv's and symlinks. An application IN use can't be altered FOR ITS CURRENT USER. But, any invocation of that application by another user can be directed around the application in question:

# mv /usr/bin/app /usr/bin/oldapp
# mv /usr/bin/newapp /usr/bin/app

At this point, any instances of app that are currently running remain the "old" version of the application. Any new instances of "app" that are started will invoke the "new" version -- even if invoked by a user who happens to have an instance of the old application currently running!

But, you knew that, right? :>

Likewise for libraries and shared libraries.

False once again, go reread how shared libraries/DLLs/anything that software shares between applications including, and that poor user who is currently using that software breaks into pieces and restart time is not trivial. Again the key is to PLAN for downtime, the users also need to PLAN for downtime, and it has to happen once in a while. Unless you've only using monolithic, static software that never has any vendor bugs?

Why? Mine work absolutely fine! But, are you sure YOURS do?

Mine work perfectly fine too. But you had to go rain down on it just because it's different than yours.

Sit down at your workstation (and don't get up cuz that will magically transform it into a SERVER -- make sure you've emptied your bladder cuz you're going to be there for a while!)

Now, assume whatever privilege level is required for your particular OS. Move to the top level directory of the source tree for your SYSTEM (not just your kernel). Type "make world" -- or its equivalent. All should go well, right? Cuz you already KNOW what to expect -- because "you should know what to expect"! Let us know how that goes for you...



You might want to go back and reread my posts, here, and decide if you want to double down on that accusation. In particular, you'll note that I HAVE NO PROBLEMS with my configuration. Years of administering systems have taught me many of the ways you can get shot in the foot -- so, I've changed my "best practices" to reduce the possibility of those sorts of issues eating my lunch. If you want to remain ignorant, go ahead and get shot! Then, YOU can have the "experience" to share!

(sigh) Sounds like I've seriously wounded YOURS! But, then again, if your only use for a "computer" is to play games and visit on-line forums, I guess it would be relatively easy to bruise... or, do you have some magical machine that you NEVER have to wait for? I guess all those two-layer boards don't require much effort to layout (or design).

Or, is it all just "The red eight goes on the black nine..."?

Yes, sigh. This is the kind of garbage people like you write that have absolutely no value to anyone here. You assume that others don't have the experience you have and crap on every point by point based on that assumption. Such people need the same response in return just to make them realize their contribution or their presentation method is unwanted. The latter applies here. Everything that you think isn't right is a nail that need to be hammered based on your oh so mighty experience that you assume nobody else has.

You should go back to playing solitare, sounds like you finally figured out how to play. You need to play badly.

So much for ranting a rant, I'm done, carry on.

[Curious.George]

Re: 10 hours lost because stupid laptop wouldn't boot! GRRR!

The key is to PLAN AHEAD.

Yes, PLAN AHEAD. Configure your system so that you DON'T HAVE TO SIT AND WATCH IT, regardless of what it MIGHT end up doing.

[examples of things that can break a system]
Seems like all you have is a hammer so everything looks like a nail. I see that's indeed what you've been doing for every post you write.

Gee, I'm at a loss to see how that comment even applies. I'll interpret you to be saying: "No, I'm completely clueless about how the system I rely upon actually works and/or can be subverted. Nor do I have any interest in finding out -- as I wouldn't know where to start."

... but yes a thousand over a few days is something that has happened over several occasions

Then, you would know how much disk space that required, right? And, how long it took? Or, did you forget those figures, already? Didn't you add them to your sysadm log?

- though the goal is to not have that happen - it means a neglected system. Did you neglect yours for the multiple thousands of packages, of which many may have been exploited during that time until the batch update?

Wow, you're showing just how constrained your thinking is!

Have you considered:
- building a NEW system and BUILDING the packages for it (instead of accepting binaries built by someone else)?
- quarterly "major releases" of the package collections (2018Q3 released on 5 Oct)?

Its no wonder you can't "imagine" the sorts of things thaty can -- and are likely -- to go wrong with a system! And, as a result, TETHER YOURSELF to yours, lest it do something unexpected while you're off taking a piss!

My machines that needed 1000-package updates were powergapped.

NONE of my machines has access to the outside world (save this one). As a result, the risk of unpatched vulnerabilities to the system's integrity is negligible.

Building everything from sources (which have had their hashes verified, automatically, as part of the process) reduces the risk that someone who built a "binary" may have had their system compromised, at the time (and that compromise propagated through the hacked, SIGNED binary).

Of course, if you're unable to understand the sources (or the algorithms involved) for the thousands of packages (17,000 in the latest release), I guess you'd have to rely on someone else to vouch for them, on your behalf. The, whine when something doesn't work quite right and hope someone has come up with a fix/workaround that you can exploit (without bothering to understand it).

But, hey, I guess all YOU really need to know is which end of the soldering iron to hold, right?

[disk space being consumed]

Yes. Or use a program to do it... Or you plan for it as you write 2000 words when only 3 are needed.

Again, showing your ignorance.

Disk space can be consumed for any number of reasons (first, you have to discover WHICH!).

Of course, programs can grow log files -- whether these are running services or "commands" that the user has directly (or indirectly) invoked. E.g., accounting track9ing every command's execution and resource utilization.

Programs can also generate "output" as part of their inherent purpose. E.g., make(1) inevitably causes new binaries to be created -- along with many intermediate files (which may require an explicit "make clean" to remove).

Programs can also create new filesystem objects as part of their purpose. E.g., rsync -avz --delete pkgsrc.se:istfiles/ /Source/distfiles/

Services can be consuming space in the filesystem. E.g., named(8) doing zone transfers; ftpd(8) accepting files from other clients; etc.

(real) Users can be adding stuff to the filesystem.

If all you see if a number (freespace0 getting smaller, that tells you nothing about its cause, how far it is likely to proceed and what you can do to stop it from reaching a point where it can be bothersome (or, if you've misconfigured your system to be vulnerable to such actions, outright dangerous!)

[But, hey, if all you're computer is doing is playing solitaire, then none of these things are llikely happening (or, at least, YOU aren't aware of them!)]

If you'd rather I trim all that to just 3 words: "showing your ignorance"

Oh great you can go change things in TWO places to make sure the both stay in sync, so much for letting the computer do that work. And go reread again, there are things you simply cannot replace a file, directory, etc., and expect everything to work shiny and new. Someone has to suffer downtime once in a while, just need to plan for it.

No. YOU reread what was written. Then, actually TRY it. But, you'll have to have a BETTER understanding of how so's work (instead of just the juvenile explanation that is intended for folks lacking interest in the details).

And, it's EXACTLY the sort of thing that you script! Because, if you kow how things work, under the hood, you know that you have all of the tools that you need to determine which libraries a particular binary relies upon. And, if you've build your system well (instead of relying on someone else to do it for you and HOPING they understand and appreciate these issues), this is all a piece of cake!

E.g., I have several versions of all of the so's my system has used, over time. Each is labeled libBLAH.major.minor.so. So, I can explicitly bind an executable to a particular version (major.minor) of a particular library.

Want a new version of app to come into being that avoids libBLAH.M.n in favor of libBLAH.X.y? Just link "newapp" to X.y explicitly. Move "oldapp" aside and put newapp in its place (as above). Next time someone invokes "app", they get "newapp" -- complete with libBLAH.X.y.

Folks who've already got instances of "(old)app" running are stuck with that because modern OS's don't allow for components to be swapped out of RUNNING binaries (the OS that I've designed DOES allow this as it has to run apps for YEARS at a time and you don't want to have to reboot or restart them just because some "component" needed to be upgraded: "Sorry, Mr. Ford, we have to shut down the assembly line while we patch the calendar routine to account for the newest batch of leap seconds announced!"

False once again, go reread how shared libraries/DLLs/anything that software shares between applications including, and that poor user who is currently using that software breaks into pieces and restart time is not trivial. Again the key is to PLAN for downtime, the users also need to PLAN for downtime, and it has to happen once in a while. Unless you've only using monolithic, static software that never has any vendor bugs?

Wow, you really are ignorant of how things work, under the hood! Do you have any formal training (e.g., degreed program) in "computer science"? Or, even just plain-old "programming"? "Operating system design"? "Language design"? Or, do you pick up everything you "know" by listening to folks at the watercooler?

If you CLAIM you have, then prove it to me -- explain the (or ANY) mechanism that I use to swap libraries out of RUNNING applications without the client being aware of the change. It's a relatively simple problem -- especially if you understand how linkage editors work and the structure of modern OS's. If you can't address even this sort of "classroom" problem, then you clearly aren't operating on anywhere near a par with me... and, thus, a waste of my time to engage you in discussions that are obviously going right over your head!

Mine work perfectly fine too. But you had to go rain down on it just because it's different than yours.

I pointed out the fault in your approach AND AN EASY WAY TO AVOID IT. If you'd rather tie yourself to a machine watching several windows that try to convey the details of what MIGHT be happening in areas of your machine that YOU CAN'T SEE, then go for it! Sure sounds like a perfect place for "use a program to do it..." (or, a configuration change that eliminates the need for that!)

Yes, sigh. This is the kind of garbage people like you write that have absolutely no value to anyone here. You assume that others don't have the experience you have and crap on every point by point based on that assumption. Such people need the same response in return just to make them realize their contribution or their presentation method is unwanted. The latter applies here. Everything that you think isn't right is a nail that need to be hammered based on your oh so mighty experience that you assume nobody else has.

I'm SHARING my experience. Because I have A LOT OF IT. And, quite obviously, in areas that you haven't BEGUN to touch. You can ignore it at your own risk/pleasure. Someone else (now or via a later search) might actually learn something from it.

You, instead, appear bent out of shape because some of your "trusted beliefs" are built on sand. And, you don't like this being exposed, publicly. That's the sort of mentality that ensures you'll never grow, intellectually -- nor emotionally. (take your ball and bat and go home)

You should go back to playing solitare, sounds like you finally figured out how to play. You need to play badly.

No games, here. Hacking together a preliminary layout (feasability) for an 8 layer PCB to "solve" a client's latest problem to see how small/cheap I can get it. But, hey, you do that all the time, right?

So much for ranting a rant, I'm done, carry on.

Oh, but I was SO looking forward to your explanation of how to replace libraries IN running applications! To PROVE you actually DO understand the drivel you've been spouting!

Or, don't -- as the case (most probably!) may be.