Expert application development!



  • There is an application I'm exposed to about twice a year. One of my customers has what is basically a fund raising drive. 50 volunteers man PCs and phones. People call in, the volunteers take a poll and enter data into a VB app, it's important to understand that this event is tied to schedules beyond our control, there are no do-overs. So the application needs to be rock solid. The front end is pretty decent, you can easily tab between fields in proper order, there is ok help and the errors make some sense. But the back end...

    Year 1 - the system isn't working correctly, the developers don't know why. They fix it, the next day, the same problems reoccur. The data files are stored on a shared network drive that all the clients access. We discover they're using the archive bit somehow to flag data files. Every night, backups run, "corrupting" their data.

    Year 2 - they've switched to a real database, MS Access. Their development testing is great, but somehow when those 50 users all login for the test, there are problems. We help them transfer the data into our SQL database  and modify the application to use it.

    Year 3 - The system keeps crashing when any volunteer logs in to it, though the developer can login fine. We discover their system stores a unique value for each user, containing the PC name + username. The database field for that value is only 15 characters. Our PC names are 12 characters, the developer's name is Bob.

    Year 4 - Since their data is used only once a year, it gets deleted by cleanup scripts. Do they have a backup... no, Do we have a backup from a year ago... luckily, yes.

    Year 5 -  A total re-write. A new database. The database contains dozens of stored procedures for no apparent reason. During testing, they find deadlocks occurring constantly. After blaming the database server (the same one they'd used for years) for a while, they modify/remove most of the procedures, upon which the system starts operating mostly normally.

    Oh yes, the other back end... 

    They have a web based reporting module. There is a special VB app which runs on a PC (complete with DO NOT TOUCH sticky note), taking data out of the database and outputting to text files. On a web server runs a number of perl scripts which massage those text files and output the data in a variety of bizarre formats, some of which are also, yes, text files. Somehow corporate downloads the text files and puts them into some display system for the monitors in the office lobbies. Year 1, the developer asks if the web server has enough horsepower to run the scripts. Turns out that yes, the Dual CPU box handles 900 hits over a 6 hour time period just fine.

    As for the rock solid part, they've never actually had a significant failure during normal operation, but they have all sorts of high end failover code just in case. For example, there are hidden keystrokes to invoke panic mode. In panic mode, the app stops using the database and sends all output to text files on the shared network drive. If that fails, there is double panic mode, if the network dies, the app (which runs off a network drive) outputs it's files to the local disk. I guess you can use sneaker-net to get the files back to the DO NOT TOUCH computer. I'm not sure what happens in triple panic mode, it may involve the bottle of scotch in the developers briefcase.

    Sadly, the people who develop is are really nice, and to be honest, have been successful (as far as corporate is concerned) every year. Technically though, seem a bit in over their heads. Corporate doesn't know or care they're flirting with disaster.

     



  • @jjeff1 said:

    they've switched to a real database, MS Access
    This made me LOL.

    Seriously, though, good job.  Well written. 



  • The access bit got a laugh out of me, too.

    This is just one of those things where a carefully crafted complete failure(that can't be traced back to you!) is a good thing. As long as the "in 5 words or less" summary that the guy at the top gets isn't "it all broke, oh poop", this will just keep going on. It's obviously going to fail one day, it just might as well be earlier to save pain down the road.



  • @jjeff1 said:

    Year 3 - The system keeps crashing when any volunteer logs in to it, though the developer can login fine. We discover their system stores a unique value for each user, containing the PC name + username. The database field for that value is only 15 characters. Our PC names are 12 characters, the developer's name is Bob.

    Year 3 postmortem notes:

    • Next year, require developers to use their full names.
    • Hire testers, preferably Indian (no one named Raj though).



  • wow, they could of just bought one (like opinio)...or get an open source one and modify it...but where's the job security in that? ;)

     

     



  • Not only that, but that doesn't show any knowledge.  Writing your own, even if it's bad, is often times viewed as better because you understand your company etc.



  • Damn it people reply to the front-page NOT the sidebar that will be resurrected when posted in.



  • @Lingerance said:

    Damn it people reply to the front-page NOT the sidebar that will be resurrected when posted in.

    No shit.  Can we please get auto-locking of threads after 7 days with no new posts?  Where would one go about making this a reality?



  • @morbiuswilters said:

    @Lingerance said:

    Damn it people reply to the front-page NOT the sidebar that will be resurrected when posted in.

    No shit.  Can we please get auto-locking of threads after 7 days with no new posts?  Where would one go about making this a reality?

     

    We need mods in general around here... But AmmoQ should be able to help.



  •  @tag said:

    ammoQ! I choose you!

     

    ammoQ casts FREEZE on Old Thread!

     

    ammoQ uses FLAMETHROWER on Lysis!

    Critical Hit!

    Lysis has fainted! 



  • @morbiuswilters said:

     @tag said:

    ammoQ! I choose you!

     

    ammoQ casts FREEZE on Old Thread!

     

    ammoQ uses FLAMETHROWER on Lysis!

    Critical Hit!

    Lysis has fainted! 

     

    LMAO

    MPS dances on the graves.


Log in to reply