Make sure it's initialized!


  • ♿ (Parody)

    @Weps said:

    Except, the dotnet GC isn't ok, it is not a solved problem. It actually causes problems.

    Do you have a reference about the sorts of problems it causes? Or even something resembling a detail?



  • @boomzilla said:

    @Weps said:
    Except, the dotnet GC isn't ok, it is not a solved problem. It actually causes problems.

    Do you have a reference about the sorts of problems it causes? Or even something resembling a detail?

    Imagine a time critical image processing application, that should not be the limiting factor for the hardware. And it should process, say, 20.000 images in 10 minutes.

    Now add a GC that itself will determine when to free memory. And, when it does, it will block threads.



  • @Weps said:

    @boomzilla said:

    @Weps said:
    Except, the dotnet GC isn't ok, it is not a solved problem. It actually causes problems.

    Do you have a reference about the sorts of problems it causes? Or even something resembling a detail?

    Imagine a time critical image processing application, that should not be the limiting factor for the hardware. And it should process, say, 20.000 images in 10 minutes.

    Now add a GC that itself will determine when to free memory. And, when it does, it will block threads.

    Uh huh...? And...? We're waiting for the second half of this amusing tale!



  • Dunno, 20,000 images in 10 minutes means 30ms per image. GC could conceivably have an impact on that timing.

    Let's wait for the conclusion of this epic mythos.

     

     



  • @pkmnfrk said:

    Uh huh...? And...? We're waiting for the second half of this amusing tale!

    Enlighten me. (referring to your hint). 



  • @Weps said:

    @pkmnfrk said:

    Uh huh...? And...? We're waiting for the second half of this amusing tale!

    Enlighten me. (referring to your hint). 

    YOU SET UP A STORY AND NEVER FINISHED IT!

    Look, so we have 20,000 images, 10 minutes, the GC decides when to free memory, and it can suspend threads to do so. That's a great set-up, now where's the FREAKING STORY!?

    If you don't provide one, I'll just assume it concludes, "and so the program ran fine with like 3 minutes to spare, man I'm an idiot why did I think the GC was bad I'm so stupid."



  • @blakeyrat said:

    @Weps said:
    @pkmnfrk said:

    Uh huh...? And...? We're waiting for the second half of this amusing tale!

    Enlighten me. (referring to your hint). 

    YOU SET UP A STORY AND NEVER FINISHED IT!

    Look, so we have 20,000 images, 10 minutes, the GC decides when to free memory, and it can suspend threads to do so. That's a great set-up, now where's the FREAKING STORY!?

    If you don't provide one, I'll just assume it concludes, "and so the program ran fine with like 3 minutes to spare, man I'm an idiot why did I think the GC was bad I'm so stupid."

    Gosh, my fault thinking I was dealing with Sherlocks over here.

    So, I guess, there is no one here who finds it abnormal that a GC blocks your application, on all threads. And in 'our' case, that was for SECONDS.... Yes, we have a few tera bytes of data to process.

    And I guess, there is no one here that finds it abnormal that you can actually have out of memory exceptions caused by a "full" GC.

    None of these issues are solved, just 'prevented'. More CPU power, more cores, more memory and a few more calls to the GC to clean up (which afaik it still only does when it wants to). 

    Neither of those two issues would have occured without a GC, or maybe with the java variant (I got no clue at to how the java gc works). And I assume they changed or fixed stuff in the umptieth dot net patch. I don't know. But I do know that if it was that good, I would not need anything to know about it.



  • @Weps said:

    So, I guess, there is no one here who finds it abnormal that a GC blocks your application, on all threads. And in 'our' case, that was for SECONDS.... Yes, we have a few tera bytes of data to process.

    And I guess, there is no one here that finds it abnormal that you can actually have out of memory exceptions caused by a "full" GC.

    None of these issues are solved, just 'prevented'. More CPU power, more cores, more memory and a few more calls to the GC to clean up (which afaik it still only does when it wants to). 

    Neither of those two issues would have occured without a GC, or maybe with the java variant (I got no clue at to how the java gc works). And I assume they changed or fixed stuff in the umptieth dot net patch. I don't know. But I do know that if it was that good, I would not need anything to know about it.

    I have worked with large systems, but never where there are "a few tera bytes of data" in memory (which is all the CG cares about) [the biggest server I have access to has 512GB]...

    With any memory management (automatic or hand coded), there will be time taken. So even if the specific issues you saw did not happen, it is likely that something else would have. So it is not a choice between "problem" and "perfection", but rathere a choice between problem domains.

    I have worked with real-time procssing of images where there is a large amount of data being processed continually. Even here it is extremely rare to have "Full" garbage collections (GEN 2), perhaps one every 2-3 days; and then they run with about 250mS to 600mS of "all threads blocked". Given that Windows is not a RTOS, there are plenty of other "normal" system conditions where application threads may not execute for even longer periods, so it has not be a determining factor.

    Based on systems I have worked on where issues such as you are reporting have occured, I have found a few common design/implementation flaws:

        * Not selecting the "right" GC engine.
        * Keeping objects around longer than necessary (triggering promotions). Even on a small machine (i-5 2.4Ghz 8GB), .NET has no problem with upwards of 5 million allocations (that distinct calls to "new", not a byte count) per second without noticable performance differential.
        * Not properly handling "large objects" [over 80KB for a single monolith].



  • @Weps said:

    @pkmnfrk said:

    Uh huh...? And...? We're waiting for the second half of this amusing tale!

    Enlighten me. (referring to your hint). 

    Depending on what you mean by "image processing", the maths would likely be slower than the GC. If you're just cropping them, then maybe not.

    @Weps said:

    Gosh, my fault thinking I was dealing with Sherlocks over here.

    So, I guess, there is no one here who finds it abnormal that a GC blocks your application, on all threads. And in 'our' case, that was for SECONDS.... Yes, we have a few tera bytes of data to process.

    And I guess, there is no one here that finds it abnormal that you can actually have out of memory exceptions caused by a "full" GC.

    None of these issues are solved, just 'prevented'. More CPU power, more cores, more memory and a few more calls to the GC to clean up (which afaik it still only does when it wants to). 

    Neither of those two issues would have occured without a GC, or maybe with the java variant (I got no clue at to how the java gc works). And I assume they changed or fixed stuff in the umptieth dot net patch. I don't know. But I do know that if it was that good, I would not need anything to know about it.

    Would you rather it doesn't suspend your thread, so that when it runs it's modifying pointers in code that is actively executing?

    If you're generating terabytes of garbage objects all the time, then yes, duh, you're going to have a lot of clean up.

    If you're generating terabytes of objects which get promoted to higher generations and then releasing them all after a bit... Well, have fun with that.



  • @TheCPUWizard said:

    @Weps said:

    So, I guess, there is no one here who finds it abnormal that a GC blocks your application, on all threads. And in 'our' case, that was for SECONDS.... Yes, we have a few tera bytes of data to process.

    And I guess, there is no one here that finds it abnormal that you can actually have out of memory exceptions caused by a "full" GC.

    None of these issues are solved, just 'prevented'. More CPU power, more cores, more memory and a few more calls to the GC to clean up (which afaik it still only does when it wants to). 

    Neither of those two issues would have occured without a GC, or maybe with the java variant (I got no clue at to how the java gc works). And I assume they changed or fixed stuff in the umptieth dot net patch. I don't know. But I do know that if it was that good, I would not need anything to know about it.

    I have worked with large systems, but never where there are "a few tera bytes of data" in memory (which is all the CG cares about) [the biggest server I have access to has 512GB]...

    With any memory management (automatic or hand coded), there will be time taken. So even if the specific issues you saw did not happen, it is likely that something else would have. So it is not a choice between "problem" and "perfection", but rathere a choice between problem domains.

    I have worked with real-time procssing of images where there is a large amount of data being processed continually. Even here it is extremely rare to have "Full" garbage collections (GEN 2), perhaps one every 2-3 days; and then they run with about 250mS to 600mS of "all threads blocked". Given that Windows is not a RTOS, there are plenty of other "normal" system conditions where application threads may not execute for even longer periods, so it has not be a determining factor.

    Based on systems I have worked on where issues such as you are reporting have occured, I have found a few common design/implementation flaws:

        * Not selecting the "right" GC engine.
        * Keeping objects around longer than necessary (triggering promotions). Even on a small machine (i-5 2.4Ghz 8GB), .NET has no problem with upwards of 5 million allocations (that distinct calls to "new", not a byte count) per second without noticable performance differential.
        * Not properly handling "large objects" [over 80KB for a single monolith].

     Well, we use 'm and 'free' 'm. Ain't much more to it.

    And of course, there will always be time taken by a MM, tho, I'ld rather have that time taken directly when an instance is no longer referenced than further up the road (when whatever makes the GC do its thing).



  • @pkmnfrk said:

    @Weps said:
    @pkmnfrk said:

    Uh huh...? And...? We're waiting for the second half of this amusing tale!

    Enlighten me. (referring to your hint). 

    Depending on what you mean by "image processing", the maths would likely be slower than the GC. If you're just cropping them, then maybe not.

    So what's the hint here? Be faster than the GC?



  • @blakeyrat said:

    Are you trying to imply that I said the Registry doesn't have any problems? Or... what is the point of this paragraph?
    In your original post, you told me that I "have to admit" that the Registry solves a number of problems and provided that link.  The Registry is like those ads where they say "Take MegaDrug to cure Athlete's Tongue!!", but then "May cause drowsiness, halitosis, dizziness, vomiting, diarrhea, frequent urination, rapid heart rate, slow heart rate, rapid breathing, slow breathing, tongue swelling, permanent nerve damage, sexual attraction to [pangolins/corpses/trees/buildings/soda bottles], suicidal or homicidal ideations."  Maybe it fixed the immediate perceived problems, but the side effects just blow chunks.  There was no reason the .INI format couldn't have been tweaked a little bit and enhanced to provide similar capabilities to the Registry.  Want enhanced permissions?  Okay, put certain settings in one .INI file and other settings in another .INI file and set the permissions as appropriate.  Allows for a hierarchical setup of configuration files and prevents corruption of a single .INI file from completely wrecking the system in the event of corruption.  Windows doesn't normally compress the allocated size of the Registry on its own; if I have a text config file that goes from 100 KB down to 20 KB (for any reason -- take your pick), I get that space back in the file system.  I don't get that space back in the Registry, even though internally the Registry really acts like a file system.  It's easy to add comments to a .INI file to understand what's going on. @blakeyrat said:
    First of all, yes they have. Netware had a Registry, for example, although I'm not sure if the most recent version still does. Strangely, Netware was also very popular for managing large corporate networks-- I'm noticing a pattern here!
    Novell was used under DOS and Win3.1 without the client needing to use a massive Windows registry.@blakeyrat said:
    Here's a fun challenge for you: you're a network administrator, and you have a network full of Macs. How do you force them to "require password on wake from sleep?" In Windows, it's trivial. In OS X? The most advanced UNIX-esque OS? Well... you can do it at install time, I guess, but if you don't do it then, then you're fucked-- you're back to the 1995 solution of walking to every single workstation, changing the setting, then locking the user account from changing it back. EFFICIENT!
    Massive fail there.  I don't know the name of the product -- it's being managed by our field technical services group -- but I set up the networking for the servers that are used to manage our corporate Macs.  And TheCPUWizard already linked to a product that could do exactly this that has been around for many years.  Next?@blakeyrat said:
    What you need to point out to me, when saying the Registry is oh so horrible, is an OS that manages to have all of the features Windows has while simultaneously not having a Registry. That thing doesn't exist. Or if it does, please show it to me, because I'd love to see it.
    Whatever I come up with, even if the feature is comparable, you're going to shoot down my examples.  So why don't you list the features that Windows provides that you feel can't even be possible without using the Registry and I'll find counterexamples to those?



  • @Weps said:

    @pkmnfrk said:
    @Weps said:
    @pkmnfrk said:

    Uh huh...? And...? We're waiting for the second half of this amusing tale!

    Enlighten me. (referring to your hint). 

    Depending on what you mean by "image processing", the maths would likely be slower than the GC. If you're just cropping them, then maybe not.

    So what's the hint here? Be faster than the GC?

    The hint here is that if you're generating a lot of garbage, then don't be surprised when Mom takes forever to clean it up.



  • @blakeyrat said:

    3) Registry functions are kept in the Microsoft namespace because they're specific to Microsoft OSes. .net is cross platform, dumbshits. (Atwood piles on to this retarded point.)
    4) .net doesn't use the Registry for ... the exact same reason as point 3. Not because .net developers "hate" the Registry, or whatever fake-ass reason they're assuming, but because .net is cross-platform and therefore can't rely on the Registry even existing at runtime.
    Okay, if we're playing the "you have to admit" game, you have to admit that, for all practical purposes, .Net is cross-platform in name only.  I know of Mono, but realistically right now anything that is .Net has to run on a Windows-based platform of some kind.  Which means there is some form of a Registry.  Which means that Microsoft is still advocating for a way to save configuration data outside of the Registry.



  • @pkmnfrk said:

    The hint here is that if you're generating a lot of garbage, then don't be surprised when Mom takes forever to clean it up.

    Ahh, the greatest fallacy...but first a diversion....

    What is the difference between how how a single woman cleans their apartment and a bachelor does? [please no posts about stereotypes]

    The answer is that the woman will spend hours cleaning up the garbage, while the backelor will take the "good stuff" (non-garbage), move to a new clean apartment and start over...

    The relationship is that people think performance of the GC is in any way related to the amount of garbage. It is not. It is directly related to the number of "live objcts" that have to be scanned and preserved.

    Reduce the number of objects that have to be scanned, and reduce the number of objects that have to be preserved (promoted), and the time taken for a GC cycle will drop tremendously.

    During training, I oeften use a parasitic example. The first condition creates a bunch of objects, and keeps them all "live". The second condition creates the exact same structure of objects, then makes them all "garbage". I ask the students which will take longer for a GC cycle, and the majority (often all) of them will reply "the one with the most garbage, of course" - and they are all wrong. When they see the "0 garbage" take nearly 100mS and the "2GB of garbage" take under 100uS, the first reaction is "your test is flawed". After a bit of study, the lightbulbs start to go on.

    (But nobody "shoude care" about this.... right?)



  • @nonpartisan said:

    @blakeyrat said:

    3) Registry functions are kept in the Microsoft namespace because they're specific to Microsoft OSes. .net is cross platform, dumbshits. (Atwood piles on to this retarded point.)
    4) .net doesn't use the Registry for ... the exact same reason as point 3. Not because .net developers "hate" the Registry, or whatever fake-ass reason they're assuming, but because .net is cross-platform and therefore can't rely on the Registry even existing at runtime.
    Okay, if we're playing the "you have to admit" game, you have to admit that, for all practical purposes, .Net is cross-platform in name only.  I know of Mono, but realistically right now anything that is .Net has to run on a Windows-based platform of some kind.  Which means there is some form of a Registry.  Which means that Microsoft is still advocating for a way to save configuration data outside of the Registry.

    What are you smoking? 1) You admitted that you know of mono, it's ON LINUX.  2) IT'S AN OPEN SPEC.  The only thing stopping there from being a version for MacOS is APPLE.


  • @Sutherlands said:

    The only thing stopping there from being a version for MacOS is APPLE FANBOIS.

    FTFY. Apple would probably do it if there were money in it. As it is, they know what their users' reaction to .NET would be: ".NET? sniff That doesn't even go with my brushed aluminum case, my MINI Cooper with the 'Think Global, Act Local' bumper sticker, or my ironic army jacket."



  • @Sutherlands said:

    What are you smoking? 1) You admitted that you know of mono, it's ON LINUX.  2) IT'S AN OPEN SPEC.  The only thing stopping there from being a version for MacOS is APPLE
    You're absolutely right.  I wasn't aware that Mono was so far advanced.  I shall eat my serving of crow.  +1 to Blakey for the cross-platform bit, but -1 to Blakey for not knowing that Macs can be centrally managed without using a Registry.



  • @nonpartisan said:

    Finally, I have had Registry corruption cause an unrecoverable workstation that required a reinstall;

    In what decade?

    @nonpartisan said:

    I've never had a corrupted .cfg file under . . . let's just say "other operating systems" . . . cause me to not be able to get it back up and running, even if I had to boot into rescue/emergency mode.

    If you had backed up your registry you should have been able to get it working again (doesn't System Restore just do that automagically now?) Just as you should back up your text config files. What you're basically saying is "I don't do backups and then I get all pissy when I lose shit."

    Not to mention, Gnome, the most common FOSS desktop environment uses a registry. Of course I wouldn't expect you to know this since you just installed Ubuntu for the first time last week.

    It seems like the entire anti-registry argument boils down to "There might be some crap in there I don't need" which is stupid, unless it's causing an actual problem. A modern Linux install has thousands upon thousands of config files crammed full of shit most people never even look at. The nice thing about a registry is that configuration settings appear in a predictable place (instead of scattered wherever the upstream project and/or distro maintainer decided) and have a good API for programmatic manipulation. Also you get strict-typing, field-level security and a consistent API that's easily understood by any programmer familiar with your platform. The downside is that it appears to be slightly more opaque than a text file, which hardly seems to be much to complain about.



  • @morbiuswilters said:

    @nonpartisan said:
    Finally, I have had Registry corruption cause an unrecoverable workstation that required a reinstall;

    In what decade?

    @nonpartisan said:

    I've never had a corrupted .cfg file under . . . let's just say "other operating systems" . . . cause me to not be able to get it back up and running, even if I had to boot into rescue/emergency mode.

    If you had backed up your registry you should have been able to get it working again (doesn't System Restore just do that automagically now?) Just as you should back up your text config files. What you're basically saying is "I don't do backups and then I get all pissy when I lose shit."

    Not to mention, Gnome, the most common FOSS desktop environment uses a registry. Of course I wouldn't expect you to know this since you just installed Ubuntu for the first time last week.

    It seems like the entire anti-registry argument boils down to "There might be some crap in there I don't need" which is stupid, unless it's causing an actual problem. A modern Linux install has thousands upon thousands of config files crammed full of shit most people never even look at. The nice thing about a registry is that configuration settings appear in a predictable place (instead of scattered wherever the upstream project and/or distro maintainer decided) and have a good API for programmatic manipulation. Also you get strict-typing, field-level security and a consistent API that's easily understood by any programmer familiar with your platform. The downside is that it appears to be slightly more opaque than a text file, which hardly seems to be much to complain about.

    "But I can't edit it in vi!"



  • @pkmnfrk said:

    @morbiuswilters said:
    @nonpartisan said:
    Finally, I have had Registry corruption cause an unrecoverable workstation that required a reinstall;

    In what decade?

    @nonpartisan said:

    I've never had a corrupted .cfg file under . . . let's just say "other operating systems" . . . cause me to not be able to get it back up and running, even if I had to boot into rescue/emergency mode.

    If you had backed up your registry you should have been able to get it working again (doesn't System Restore just do that automagically now?) Just as you should back up your text config files. What you're basically saying is "I don't do backups and then I get all pissy when I lose shit."

    Not to mention, Gnome, the most common FOSS desktop environment uses a registry. Of course I wouldn't expect you to know this since you just installed Ubuntu for the first time last week.

    It seems like the entire anti-registry argument boils down to "There might be some crap in there I don't need" which is stupid, unless it's causing an actual problem. A modern Linux install has thousands upon thousands of config files crammed full of shit most people never even look at. The nice thing about a registry is that configuration settings appear in a predictable place (instead of scattered wherever the upstream project and/or distro maintainer decided) and have a good API for programmatic manipulation. Also you get strict-typing, field-level security and a consistent API that's easily understood by any programmer familiar with your platform. The downside is that it appears to be slightly more opaque than a text file, which hardly seems to be much to complain about.

    "But I can't edit it in vi!"

    vi can edit anything: you're just not trying hard enough. Also: there are CLI tools for editing the Registry for Windows and Linux, if for some reason you are afraid of GUIs.



  • @morbiuswilters said:

    In what decade
    Last decade.  Windows XP.  Generic workstation in the emergency department.  Based on a generic build so it wasn't difficult to restore.  No permanent data storage -- used for reading radiology films, patient record access, etc.  Still, took a couple of hours.  On a machine that used text config files I'd have had the machine back up in less time than that without a full reinstall. @morbiuswilters said:
    Not to mention, Gnome, the most common FOSS desktop environment uses a registry. Of course I wouldn't expect you to know this since you just installed Ubuntu for the first time last week.
    I don't have a problem with a "registry" per se, with a common API to store settings.  I have a problem with the Windows Registry, a binary blob of data that can get totally fucked if one sector gets corrupted, potentially causing the entire machine to be unbootable.  This is the context from which my other replies should be taken.  Anyone who understands the idea of "context" should have understood I was talking about the Windows implementation of a registry (thus always capitalizating the "R" to refer to the Windows Registry).  No, I don't think the binary blob method was necessary.  Yes, I believe they could have enhanced the .INI file API and achieved the same effect.  That said, the GNOME registry is all text-based; if I need to copy in a section from a different machine, I could do it using cut-and-paste between two terminal windows.@morbiuswilters said:
    It seems like the entire anti-registry argument boils down to "There might be some crap in there I don't need" which is stupid, unless it's causing an actual problem. A modern Linux install has thousands upon thousands of config files crammed full of shit most people never even look at. The nice thing about a registry is that configuration settings appear in a predictable place (instead of scattered wherever the upstream project and/or distro maintainer decided) and have a good API for programmatic manipulation
    Your example of GNOME proves my point . . . you do not need to have a binary-based Registry to be able to store settings.  It's all text-based (XML).  The hierarchy can be represented in the regular file system.  And even then, this applies to the user environment.  If I have a problem with my GNOME profile that I can't figure out, I can wipe it away (like wiping away the profile directory under Windows) and have it recreated.  Yes, I lose all my settings -- just like I lose those settings under my Windows profile if I do that.  And yet, the base system can keep running.  And if a config file in my /etc directory becomes corrupted, I can edit it or copy a replacement over.  If my base system Registry gets corrupted, I'm fucked.  That may not be as true a case with Windows 7 -- I've not had my Registry corrupt under it.  But my experience with XP Registry problems was not pleasant.



  • @blakeyrat said:

    Wow Atwood's a dumbass and so are his readers.

    This is the guy who thought salting passwords meant adding the word "salt" to every password before hashing it and told all of his readers to do that. I also remember some article where he said the only way to get SQL Server to work was to use read uncommitted for all queries. I believe this is when he was doing the coding for Stack Exchange.



  • @morbiuswilters said:

    Also you get strict-typing
    Not really. Registry types are little more than hints - nothing prevents you from reading a DWORD as a string and vice-versa, since registry only stores binary blob and it's length (which means that strings can have embedded nuls).



  • @nonpartisan said:

    On a machine that used text config files I'd have had the machine back up in less time than that without a full reinstall.
    You probably could do the same with XP - there are backup copies of registry hives in C:\System Volume Information. Been there, done that.



  • @nonpartisan said:

    @morbiuswilters said:
    In what decade
    Last decade.  Windows XP.  Generic workstation in the emergency department.  Based on a generic build so it wasn't difficult to restore.  No permanent data storage -- used for reading radiology films, patient record access, etc.  Still, took a couple of hours.  On a machine that used text config files I'd have had the machine back up in less time than that without a full reinstall.

    So you aren't backing up and now you're complaining like this is somehow M$'s fault. Let's all take a moment to bask in your competency.

    @nonpartisan said:

    Yes, I believe they could have enhanced the .INI file API and achieved the same effect.

    Yeah, let's endlessly extend a shitty, outdated file format.

    @nonpartisan said:

    That said, the GNOME registry is all text-based; if I need to copy in a section from a different machine, I could do it using cut-and-paste between two terminal windows.

    So "allowing end users to copy-and-paste the entire registry instead of using the correct tool for the job" is now a reason to make shitty technical decisions?

    @nonpartisan said:

    Your example of GNOME proves my point . . . you do not need to have a binary-based Registry to be able to store settings.  It's all text-based (XML).  The hierarchy can be represented in the regular file system.

    An RDBMS can also eschew binary data and just store everything in XML. Obviously, XML is the best format for everything. (And are you seriously editing hundreds of lines of XML in a text editor or are you using an actual tool for the job?)

    @nonpartisan said:

    And if a config file in my /etc directory becomes corrupted, I can edit it or copy a replacement over.

    So your DR plans consist of "look through the thousands of text files and try to find the one corrupted, then fix it by hand".

    Don't you think there are better uses for your 1 hour of free Internet access at the public library than trolling a community of professionals? There's probably some charity group in your area with a website you can scam. Or maybe you can look up how to cook meth using Tylenol Cold and Sinus, Coleman fuel and an empty 2-liter bottle? It seems like the short-lived thrill you get from wasting my time with your retarded comments is nothing compared to the rush you'd get from some fresh-cooked scante.



  • @ender said:

    @morbiuswilters said:
    Also you get strict-typing
    Not really. Registry types are little more than hints - nothing prevents you from reading a DWORD as a string and vice-versa, since registry only stores binary blob and it's length (which means that strings can have embedded nuls).

    I think most strictly-typed systems have ways to avoid the type system so they only provide "hints" as well.



  • @ender said:

    @nonpartisan said:
    On a machine that used text config files I'd have had the machine back up in less time than that without a full reinstall.
    You probably could do the same with XP - there are backup copies of registry hives in C:\System Volume Information. Been there, done that.

    Unless you disable System Restore, been there, done that



  • @morbiuswilters said:

    I think most strictly-typed systems have ways to avoid the type system so they only provide "hints" as well.
    I'd imagine a strongly-typed system to require at least a flag if you're reading something in a different format - with Registry you just call a read function for a different format, and you'll get the data in that format.
    @serguey123 said:
    Unless you disable System Restore, been there, done that
    In that case, there are saved versions of the hives, which are usually enough to bring up the system without doing a reinstall.



  • @morbiuswilters said:

    < blah blah blah fucking stupid "answers" blah blah blah>

    @morbiuswilters said:

    An RDBMS can also eschew binary data and just store everything in XML. Obviously, XML is the best format for everything. (And are you seriously editing hundreds of lines of XML in a text editor or are you using an actual tool for the job?)
    Quote me where I said that XML is always the best format.  Go ahead, I'm waiting.  In the meantime, an RDBMS file corruption does not take down the whole fucking machine, preventing me from using the basic maintenance tools available for the OS.  A Registry corruption can, and has, done that.@morbiuswilters said:
    community of professionals?
    You're right.  My apologies to TheCPUWizard, snoofle.@morbiuswilters said:
    Or maybe you can look up how to cook meth using Tylenol Cold and Sinus, Coleman fuel and an empty 2-liter bottle?
    The fact that you know this much about cooking meth shows you know more about it than me.  Thankfully you're moving off the mainland.



  • @nonpartisan said:

    In the meantime, an RDBMS file corruption does not take down the whole fucking machine, preventing me from using the basic maintenance tools available for the OS.

    You're right, it just makes the machine unusable for it's primary purpose until the database is recovered. Which is why people make backups and have a DR plan. You keep ignoring this point, by the way.

    @nonpartisan said:

    @morbiuswilters said:
    community of professionals?
    You're right.  My apologies to TheCPUWizard, snoofle.

    Don't forget dhromed. He's a professional*, too.

    @nonpartisan said:

    @morbiuswilters said:
    Or maybe you can look up how to cook meth using Tylenol Cold and Sinus, Coleman fuel and an empty 2-liter bottle?
    The fact that you know this much about cooking meth shows you know more about it than me.  Thankfully you're moving off the mainland.

    I'm glad I'm moving, too, although I don't know why that matters to you. Does the fact that I know basic chemistry and drug policy frighten you? Do you look down on DEA agents for knowing about drugs?



  • @nonpartisan said:

    In the meantime, an RDBMS file corruption does not take down the whole fucking machine, preventing me from using the basic maintenance tools available for the OS

    sudo cat /dev/random > /etc/inittab

    Luckily, that won't corrupt a Linux system so that it won't boot* oh whoops.



  • @pkmnfrk said:

    @nonpartisan said:
    In the meantime, an RDBMS file corruption does not take down the whole fucking machine, preventing me from using the basic maintenance tools available for the OS

    sudo cat /dev/random > /etc/inittab

    Luckily, that won't corrupt a Linux system so that it won't boot* oh whoops.

    Boot a rescue disk, replace /etc/inittab, reboot.  15 minutes.

     



  • @nonpartisan said:

    @morbiuswilters said:
    community of professionals?
    You're right.  My apologies to TheCPUWizard, snoofle.
    Aw... if I start posting stuff about my work, can I be a professional, too?@morbiuswilters said:
    Do you look down on DEA agents for knowing about drugs?

    Not for that reason, no.


  • @morbiuswilters said:

    @nonpartisan said:
    In the meantime, an RDBMS file corruption does not take down the whole fucking machine, preventing me from using the basic maintenance tools available for the OS.

    You're right, it just makes the machine unusable for it's primary purpose until the database is recovered. Which is why people make backups and have a DR plan. You keep ignoring this point, by the way.

    TDWTF poll:  how many people here keep backups of their workstation down to the OS level on a day-to-day basis?  Not just data, but OS and configuration files.  Regularly.  No one?  Anyone?  Everyone?



  • @nonpartisan said:

    TDWTF poll:  how many people here keep backups of their workstation down to the OS level on a day-to-day basis?  Not just data, but OS and configuration files.  Regularly.  No one?  Anyone?  Everyone?
    I pretty much have nothing backed up.  I haven't had a disk failure yet, so haven't been bitten.  *Knock on wood*  I've been meaning to back up stuff that's important to a web drive somewhere...



  • @nonpartisan said:

    @pkmnfrk said:

    @nonpartisan said:
    In the meantime, an RDBMS file corruption does not take down the whole fucking machine, preventing me from using the basic maintenance tools available for the OS

    sudo cat /dev/random > /etc/inittab

    Luckily, that won't corrupt a Linux system so that it won't boot* oh whoops.

    Boot a rescue disk, replace /etc/inittab, reboot.  15 minutes.
    Or if I somehow stumble on it while the OS is still running, then I can fix it and no interruption in service.  How about if a Registry file corrupts in the middle of normal operation, either by strange accident or by malicious activity?

     


  • Discourse touched me in a no-no place

    @nonpartisan said:

    TDWTF poll:  how many people here keep backups of their workstation down to the OS level on a day-to-day basis?  Not just data, but OS and configuration files.  Regularly.  No one?  Anyone?  Everyone?
    Does a VM image for the purposes of escrow, backed up for every deposit, count?



  • @nonpartisan said:

    Boot a rescue disk, replace /etc/inittab, reboot.  15 minutes.
    Guess what? It works the same way with corrupted registry!
    @nonpartisan said:
    TDWTF poll:  how many people here keep backups of their workstation down to the OS level on a day-to-day basis?  Not just data, but OS and configuration files.  Regularly.  No one?  Anyone?  Everyone?
    I've got a daily backup of my system drive (which contains the OS, all installed programs and all documents I care about) scheduled for every morning at 9:20 (the backup goes to my server, where it's stored on a RAID). I keep a month of backups (and they proved useful several times when I deleted stuff accidentally).



  • @nonpartisan said:

    TDWTF poll:  how many people here keep backups of their workstation down to the OS level on a day-to-day basis?  Not just data, but OS and configuration files.  Regularly.  No one?  Anyone?  Everyone?

    My workstations/desktop/laptop machines? None. They're not important.

    My servers? All. Production data plus config files, so I'm able to recover them in the event of a disaster (or migrate if needs be).



  • @nonpartisan said:

    @morbiuswilters said:

    @nonpartisan said:
    In the meantime, an RDBMS file corruption does not take down the whole fucking machine, preventing me from using the basic maintenance tools available for the OS.

    You're right, it just makes the machine unusable for it's primary purpose until the database is recovered. Which is why people make backups and have a DR plan. You keep ignoring this point, by the way.

    TDWTF poll:  how many people here keep backups of their workstation down to the OS level on a day-to-day basis?  Not just data, but OS and configuration files.  Regularly.  No one?  Anyone?  Everyone?

    Complete differentials of each system at 3:00AM, Complete images every Sunday. Staggered retention policy [every week for a quarter, every month for a year, etc.]. Important systems (like my primary dev box) have shadows on standby, so I can fire them up even if the hardware is totally distroyed.

    I have had system failures, as well a a major disaster (house fire) a few years ago - nothing has been lost.



  • I'm not going to comment on individual backup policies yet until a decent sample has come in.  However, with respect to:@ender said:

    @nonpartisan said:
    Boot a rescue disk, replace /etc/inittab, reboot.  15 minutes.
    Guess what? It works the same way with corrupted registry!
    If you have a copy of the individual system's Registry file, then yes.  If you don't have a copy of it, then you're screwed.  But I can take any other /etc/inittab from any other like machine, throw it in, and have the system back up and running.  Or I can quickly write my own.  If it's a heavily-customized inittab, then yes, I should have a backup of it.  But if it's a stock inittab, I copy it over from any other like system and reboot.  Unfortunately, with the other settings that are stored in the base Registry files, you can't just copy one over from another machine.  At best, you'll end up with mixed configuration settings that need to be sorted out after the system is limping along.  At worst, it still won't work.



  • @nonpartisan said:

    I'm not going to comment on individual backup policies yet until a decent sample has come in.  However, with respect to:@ender said:

    @nonpartisan said:
    Boot a rescue disk, replace /etc/inittab, reboot.  15 minutes.
    Guess what? It works the same way with corrupted registry!
    If you have a copy of the individual system's Registry file, then yes.  If you don't have a copy of it, then you're screwed.  But I can take any other /etc/inittab from any other like machine, throw it in, and have the system back up and running.  Or I can quickly write my own.  If it's a heavily-customized inittab, then yes, I should have a backup of it.  But if it's a stock inittab, I copy it over from any other like system and reboot.  Unfortunately, with the other settings that are stored in the base Registry files, you can't just copy one over from another machine.  At best, you'll end up with mixed configuration settings that need to be sorted out after the system is limping along.  At worst, it still won't work.

    If you are running XP, do this. For Windows 7, the process is very similar, except the files are in C:\Windows\System32\config\RegBack. A backup is automatically taken on every successful boot.



  • @Jaime said:

    If you are running XP, do this. For Windows 7, the process is very similar, except the files are in C:\Windows\System32\config\RegBack. A backup is automatically taken on every successful boot.
    This is a procedure that I didn't have at the time.  Now I wish I did.  I will concede that there are automatic backups of the Registry files available and it doesn't look too complicated to restore them.  I could say something about "well, what if the backup is corrupted", but if both the primary Registry files and the backups are corrupted you're likely looking at a pretty screwed system (more akin to if I lost the /etc directory completely).  I would still argue that the procedure is more complicated than just rewriting /etc/inittab and being back up, but at least it can be done and still faster than reimaging the machine.



  • @nonpartisan said:

    TDWTF poll:  how many people here keep backups of their workstation down to the OS level on a day-to-day basis?  Not just data, but OS and configuration files.  Regularly.  No one?  Anyone?  Everyone?

    I have daily full backups to an external disk. I keep the last 2. Once btrfs becomes less of a piece of shit, I'm going to use it to do 15-minute snapshots which will be uploaded to my EC2 server and stored in S3. Unfortunately, right now, Linux has pretty mediocre backup support.

    For servers: I do 15-minute snapshots of everything.

    The fact that you consider backing up to be so esoteric shows your level of competence. I have made huge errors with backing up; when I got a new machine I was too busy to set up proper backups. When my SSD died without warning I lost a lot of valuable data. However, that my was my own stupid fault and I wouldn't sit around blaming my OS because I was failed to make backups.



  • @morbiuswilters said:

    The fact that you consider backing up to be so esoteric shows your level of competence. I have made huge errors with backing up; when I got a new machine I was too busy to set up proper backups. When my SSD died without warning I lost a lot of valuable data. However, that my was my own stupid fault and I wouldn't sit around blaming my OS because I was failed to make backups.
    But you see, here is where you're making assumptions about me.  Do I have my workstation backed up?  Not the OS and programs -- and initial returns show that I'm not the only one.  Do I have all the data backed up?  Everything that is important to me, yes.  I throw things onto the desktop for convenient access sometimes but I don't put things on there that I don't want to lose.  I have a home directory on a network file share.  Yes, my data is backed up.

    What you apparently didn't bother to pay attention to in my previous posts:  (a) there was no permanent data stored on the workstation that corrupted (it was a general-access workstation for MDs, RNs, etc.), and (b) I had a build image that restored the machine in two hours.  It had all the programs on it that it needed.  The worst problem for it was that the patch level was out of date, so it had to download and apply many of the patches.  The base OS and applications were installed in about 30 minutes.  Nevertheless, I still stand by the idea that if the Registry used a text-based system and organized within the file system, I'd have been able to compare it to an identical machine sitting on the other side of this one and rebuilt the corrupted config.

    I'm sure you're going to rip me to shreds on this one.  Nevertheless, one thing about me that I've not really said and you probably don't realize:  I come from a time when I had the freedom to tinker with and tweak my machine how I wanted to, bare metal, no hypervisors or virtual machines or virtualization or whatever getting in the way.  I've blown away floppy disks messing with the FAT.  I've hand-analyzed FAT and NTFS and ext2/3.  I ran a bulletin board system with text configurations.  I've hand-coded ANSI text files.  I've written software.  I've been burned many times by programs that purport to be the interface to the back end, then when there was a feature I needed, that interface was buggy, or didn't support the change I needed, or it was difficult to get to, or . . . take your pick, but the result was frustration that the program didn't do what I needed it to do.  So I would figure out the file format (either binary or ASCII), hack it, and move on.  Regardless of what you think of any of this, what it did was instill a desire to know and understand what's going on behind the scenes and be able to manipulate it.  Windows (and Linux, and FreeBSD, and OS X, and BeOS, and . . .) are so big, hard drives are so big, etc. that it is difficult to fully know and understand exactly what's going on down at the "component level", per se.  I mean, I know how most of it works in concept, but I used to understand in a practical, operational, down-to-the-hardware-component level.  So if a Registry corrupts, I have no idea the "why"s or "what"s or anything like that; it's a big binary blob that I can't troubleshoot.  But if my UNIX-type system corrupts, I have a good shot at being able to log into it, determine what went wrong, and fix the individual component that corrupted.  It enhances my understanding of what's going on and I learn from it.  Typically I also get clues about what went wrong so I can make a better judgment call -- am I experiencing a hardware failure?  Am I looking at a bug in the kernel or the application?  Your opinion comes across as "Who the fuck cares why it happened?  Just put it back in place and move on!"  I don't like the idea of losing potential information about the cause of the problem.  But I guess you young 'uns don't care about that, so fine.  Whatever.



  • @Jaime said:

    If you are running XP, do this. For Windows 7, the process is very similar, except the files are in C:\Windows\System32\config\RegBack. A backup is automatically taken on every successful boot.

    A backup of what is taken on every boot? The procedure outlined isn't simply "copy each hive file back into place" (that would be too easy), it's pages of nonsense including needing to smash your way into System Volume Information. The files that you copy back into place were created during OS installation, not boot. At least for XP, according to the above. Last Known Good Configuration assumes that the SYSTEM hive isn't toast. Maybe 7 actually does take a proper backup.

    A novel way to corrupt the Registry: mouse isn't working for some reason, therefore, switch PC off and back on. Mouse still isn't working, switch PC off and back on. Wait … the SYSTEM hive is now destroyed. (I was going to reinstall this PC, but for kicks I'll try the above behemoth of a procedure and see what happens. Edit: no this isn't my PC and I didn't do that. Brainworm prose, sorry.)

    There I thought that the Registry was transactional, but obviously in XP it wasn't protected against people who don't know how to operate Windows without the mouse. Which, sadly, is pretty much everyone. I'm not hot on keyboard support in 7, either – the user account picture control panel (as trivial as it is) doesn't even draw a focus ring. But that's off-topic … That said, I haven't taken the complaint of Registry corruption seriously since Internet Explorer 4 (?) supposedly corrupted it. There are other reasons why I dislike it, but (un)reliability is not one of them.



  • @Daniel Beardsmore said:

    A backup of what is taken on every boot?

    Windows 7 takes a backup of all of the non-user hives of the registry on every boot. Yes, you just copy them back into place. Last Known Good is a different thing, it's a subset of SYSTEM, so it requires the system hive to be accessible.

    BTW, to fix the mouse condition you described, go back to a registry backup and then do a LastKnownGood boot. You've got two problems (mouse and bad registry), so you need two fixes.



  • @nonpartisan said:

    @morbiuswilters said:

    The fact that you consider backing up to be so esoteric shows your level of competence. I have made huge errors with backing up; when I got a new machine I was too busy to set up proper backups. When my SSD died without warning I lost a lot of valuable data. However, that my was my own stupid fault and I wouldn't sit around blaming my OS because I was failed to make backups.
    But you see, here is where you're making assumptions about me.  Do I have my workstation backed up?  Not the OS and programs -- and initial returns show that I'm not the only one.  Do I have all the data backed up?  Everything that is important to me, yes.  I throw things onto the desktop for convenient access sometimes but I don't put things on there that I don't want to lose.  I have a home directory on a network file share.  Yes, my data is backed up.

    What you apparently didn't bother to pay attention to in my previous posts:  (a) there was no permanent data stored on the workstation that corrupted (it was a general-access workstation for MDs, RNs, etc.), and (b) I had a build image that restored the machine in two hours.  It had all the programs on it that it needed.  The worst problem for it was that the patch level was out of date, so it had to download and apply many of the patches.  The base OS and applications were installed in about 30 minutes.  Nevertheless, I still stand by the idea that if the Registry used a text-based system and organized within the file system, I'd have been able to compare it to an identical machine sitting on the other side of this one and rebuilt the corrupted config.

    I'm sure you're going to rip me to shreds on this one.  Nevertheless, one thing about me that I've not really said and you probably don't realize:  I come from a time when I had the freedom to tinker with and tweak my machine how I wanted to, bare metal, no hypervisors or virtual machines or virtualization or whatever getting in the way.  I've blown away floppy disks messing with the FAT.  I've hand-analyzed FAT and NTFS and ext2/3.  I ran a bulletin board system with text configurations.  I've hand-coded ANSI text files.  I've written software.  I've been burned many times by programs that purport to be the interface to the back end, then when there was a feature I needed, that interface was buggy, or didn't support the change I needed, or it was difficult to get to, or . . . take your pick, but the result was frustration that the program didn't do what I needed it to do.  So I would figure out the file format (either binary or ASCII), hack it, and move on.  Regardless of what you think of any of this, what it did was instill a desire to know and understand what's going on behind the scenes and be able to manipulate it.  Windows (and Linux, and FreeBSD, and OS X, and BeOS, and . . .) are so big, hard drives are so big, etc. that it is difficult to fully know and understand exactly what's going on down at the "component level", per se.  I mean, I know how most of it works in concept, but I used to understand in a practical, operational, down-to-the-hardware-component level.  So if a Registry corrupts, I have no idea the "why"s or "what"s or anything like that; it's a big binary blob that I can't troubleshoot.  But if my UNIX-type system corrupts, I have a good shot at being able to log into it, determine what went wrong, and fix the individual component that corrupted.  It enhances my understanding of what's going on and I learn from it.  Typically I also get clues about what went wrong so I can make a better judgment call -- am I experiencing a hardware failure?  Am I looking at a bug in the kernel or the application?  Your opinion comes across as "Who the ----  cares why it happened?  Just put it back in place and move on!"  I don't like the idea of losing potential information about the cause of the problem.  But I guess you young 'uns don't care about that, so fine.  Whatever.

    No one will accuse me of being a "young'un" having been programming since right around the introduction of 8" floppies back in the early 1970s.... You raise some good points....but....the registry format is not difficult, so anyone who has looked at NTFS (or even FAT) structures, would have no problem in "byte editing" the information. I had resorted to it a number of times. If you dont want to do that, then copy the CORRUPTED registry to a temporary location on a good machine, use the API, and then copy the file back... I did both of these quite a few times in the late 1990's, and once or twice more recently (I now tend to send clients machines that are "trashed" out to recovery houses when necessary - they do it well, and I have better things to do (i.e. they won't pay what I would charge to do it personally).



  • I'm mobile now and not going to try to quote just the part I want. I had thought about writing a file system module for Linux that would mount a hive. It is indeed very similar to a file system. However, when I last looked (guessing 2007-ish; definitely before Vista, so probably earlier) the documentation that I could find was incomplete and somewhat scattered. Maybe I'll look into it again. An interesting and enlightening project anyway.


Log in to reply