Produce a version of PostgreSQL, or a database that's fully compatible with PostgreSQL, which can be scaled for performance by simply adding more computers to the system, and making very minor configuration changes. No application changes should be necessary. Within reason, all features of PostgreSQL 8.3 must be fully supported. The relative time complexity of any particular query (either readers or writers) should be proportional to no more than log(n)/n where n is the number of identical computers running the database. The required number of network cables should be proportional to no more than n*log(n). The maximum storage capacity of the database must at least remain constant as a function of n. The implementation may assume that the number of database client machines (eg, web servers) is roughly proportional to the number of database server machines, and that the traffic from the clients is nicely balanced. If this task is impossible, a rigorous proof of that fact would be very much appreciated.

Done.  Where can I collect my $1000?

morbiuswilters:

Done.  Where can I collect my $1000?

One of the requirements seems to be that any additional machines in the cluster be fully explioted by any query. I can assure you that this is not the case for MySQL clusters, and each added machine won't necessarily provide the same performance boost as the last.

Also, the assertion that MySQL contains all features of PostgreSQL is pretty brave.

drinkingbird:

One of the requirements seems to be that any additional machines in the cluster be fully explioted by any query. I can assure you that this is not the case for MySQL clusters, and each added machine won't necessarily provide the same performance boost as the last.

That's true of any distributed system as communication overhead grows.  Still, MySQL cluster scales reasonably well, much better than Posgr-- oh really, it doesn't have any built-in clustering or replication?  And all 3rd party options suck vast quantities of buttholes?

drinkingbird:

Also, the assertion that MySQL contains all features of PostgreSQL is pretty brave.

Well, it did say "within reason", but I'll be generous here.  We better implement "vacuum maintenance" that makes the server near inoperable for hours on end, a godawful query planner that can't optimize itself out of a full table scan half of the time, slow connect times due to the lack of threading and connection pooling, and built-in limits on the number of queries that can be executed before table corruption occurs.  Obviously MySQL is used by Google because nobody who works there is as smart as you.  Seriously, though, Postgres is a great choice if your business doesn't have to be up more than 18 hours a day.  In fact, if a terrorist plants a bomb in your office and demands that your company shut down for at least 6 hours a night, Postgres is my first recommendation.  You also might consider dropping that expensive Internet connection and just send correspondence through a horseback rider.

drinkingbird:

Filed under: mysql is still a shit DBMS

You're just cranky because you have to work with inferior software all day.  My condolences.  Have a sit and a drink and then head back to your "enterprise database" that I'm sure supports dozens of clients located all over the same town! 

morbiuswilters:

That's true of any distributed system as communication overhead grows.  Still, MySQL cluster scales reasonably well, much better than Posgr-- oh really, it doesn't have any built-in clustering or replication?  And all 3rd party options suck vast quantities of buttholes?

Slony seems to be working fine. No, it doesn't have any built in clustering. So what? Maybe that's just decent separation of concerns.

morbiuswilters:

We better implement "vacuum maintenance" that makes the server near inoperable for hours on end, a godawful query planner that can't optimize itself out of a full table scan half of the time, slow connect times due to the lack of threading and connection pooling, and built-in limits on the number of queries that can be executed before table corruption occurs.

Care to provide any references to that. Honestly, I can't find anything to back that up.

morbiuswilters:

Obviously MySQL is used by Google because nobody who works there is as smart as you.

MySQL does fit the "just chuck more hardware at it" approach that Google seems to favour. Fine if that works for them.

morbiuswilters:

You're just cranky because you have to work with inferior software all day.  My condolences.  Have a sit and a drink and then head back to your "enterprise database" that I'm sure supports dozens of clients located all over the same town! 

Yeah, Postgres is definitely the "enterprise database" in this situation. The fact that it has a more permissive and straightforward licence, and isn't marketed by a for-profit company really backs that up.

If I was going to post a reply, I'd say, dude not for under a thousand bucks.  As if.

drinkingbird:

...

Holy crap. This sounds just like Russ reasoning (read: trolling). Could this be another of his online identities?

MasterPlanSoftware:

drinkingbird:

...

 

Holy crap. This sounds just like Russ reasoning (read: trolling). Could this be another of his online identities?

Your petty abuse really has declined in quality.

I don't suppose you actually have any reasons for calling labelling my posts trolling, do you?

Oh hang on. Is this one of those occasions where everyone's just going to fail to recognise your incredibly intellectual brand of humour, forcing you to point out that you were joking after you're lambasted for what is so terribly misinterpreted as blinkered ignorance and ill manners?

drinkingbird:

I don't suppose you actually have any reasons for calling labelling my posts trolling, do you?

Yes. Please read your replies.

You: I think Postgre is the best. Nothing can beat it. MySQL sucks.

Morbius: You are clearly wrong, and your ideas are ridiculous. [insightful facts that can actually be backed up]

You: [More rumor, conjecture, and pure opinion that cannot be backed up, which is why you haven't.]

This is only further backed up by your history of trolling, but even putting that aside... What is your point here?

MasterPlanSoftware:

You: I think Postgre is the best. Nothing can beat it. MySQL sucks.

Actually, my first conjecture was that mysql doesn't implement all the features of postgres, which seems pretty reasonable.Would you consider mysql and postgresql to be transparently interchangable for all purposes?

I also said that mysql clustering doesn't provide a linear increase in performance, which is true. I didn't claim anything else could do that, only that mysql wouldn't satisfy that particularly insane requirement.

I asked for some sort of backup for the performance issues mentioned for postgres. I'm aware that the vacuuming tables has been a small issue in the past, but I haven't seen anything of the sort mentioned recently, particularly with the improvements made between v6 and v8 specifically aimed at making vacuuming a nicer procedure with less degradation of overall performance. Maybe morbius is constantly using "vacuum full" which is a rather aggresive version of the function when he shouldn't (http://www.postgresql.org/docs/8.3/static/routine-vacuuming.html).

MasterPlanSoftware:

Morbius: You are clearly wrong, and your ideas are ridiculous. [insightful facts that can actually be backed up]

I've addressed these so called "insightful facts" and asked for backup. Apparently when I don't provide documentation, all my comments can be freely dismissed, but other's comments are to be taken as indisputable truth.

MasterPlanSoftware:

This is only further backed up by your history of trolling, but even putting that aside... What is your point here?

My point was just that mysql didn't satisfy all the requirements, particularly the ones satisfied by no DBMS, clustered or otherwise.

My history of trolling apparently entirely consists of times I disagreed with you. I don't see how that automatically qualifies as trolling. Your habit of feigning misunderstood humour when proven wrong is, however, backed up by your posts on this forum.

drinkingbird:

Slony seems to be working fine. No, it doesn't have any built in clustering. So what? Maybe that's just decent separation of concerns.

Slony is implemented through triggers.  Not only is this disgusting, but it is incredibly slow.  MySQL can have dozens of slaves reading off one master with only one socket for each slave.  Additionally, the data is transferred in an efficient binary format.  Now for clustering, MySQL has the ability to run a shared-nothing cluster with as many replicated partitions as you want.  The hardware can be quite expensive, but it's a rock-solid high-availability solution that was originally designed for telecom switching systems that cannot accept any downtime.  I guess someone who has used Postgres for awhile will think that "high availabilty" is a concern separate from their RDBMS, since Postgres doesn't support any HA features.

drinkingbird:

Care to provide any references to that. Honestly, I can't find anything to back that up.

Backup on what part?  Vacuums are slow and vac fulls are even worse.  And yes, you do sometimes have to run vac fulls because otherwise disk space will eventually run out.  Each time you run vacuum, it marks space as reusable in the table but doesn't return it to the OS.  If you are doing a very large number of transactions, then you will find yourself slowing running out of disk space no matter how many vacuums you do.  And forget autovacs in any real situation: the daemon will decide the table needs a vac at the most idiotic time and end up doing maintenance that grinds the DB to a halt at 11AM on a Monday morning.

I don't have any backup on the query planner, other than my own experience.  With identical tables, identical data loaded and identicial queries MySQL is consistently better at choosing an optimal query plan.  If you experience is different, bully for you, but in my experience Postgres is one of the most inefficient databases out there.

The slow connect times are well documented.  Postgres forks a new process for each connection, MySQL uses a pool of worker threads that are cached across connections.  This means MySQL already has the memory allocated, the data structures resident in memory and the thead created when a new socket is opened.  Postgres has to fork an entire process that lives only as long as that query.  What's more, this is no big secret, because Postgres has several 3rd party connection pooling utilities to prevent the process from being discarded for each connection.  Seriously, Postgres loses the scalability war hard.  I have never seen a Postgres server that could handle over 10,000 concurrent connections and 50,000 queries per second.

The table corruption thing is pretty well-known, too.  You have to run vac to reclaim oids or else postgres will just do integer wrap-around and start reusing the old ids.  This is well-documented on the vacuum page.  The means if you do a vacuum, you are limited to around 3-4 billion transactions until wraparound occurs.  In many high-volume situations, this is not acceptable.

drinkingbird:

MySQL does fit the "just chuck more hardware at it" approach that Google seems to favour. Fine if that works for them.

Yeah, I hate horizontal scalability as well.  I also forgot to mention Yahoo, Facebook, LiveJournal, etc..  What large site uses Postgres?  *crickets chirping*  Oh, I guess those people just don't know how awesome Postgres is, that's it!

drinkingbird:

Yeah, Postgres is definitely the "enterprise database" in this situation. The fact that it has a more permissive and straightforward licence, and isn't marketed by a for-profit company really backs that up.

Oh shit, somebody tell Oracle, Microsoft and Teradata that they make toy databases!  It's true, I read it on the Internet!!  Seriously, the fact that it's made by a bunch of nerds living in their parents' basements instead of a for-profit corporation makes it the more enterprise solution?  I'm not even using "enterprise" saracastically, I mean large companies that need uptime and scalability.

Honestly, my first reply with the link to MySQL was mostly meant as a joke and your response to me was so retarded I considered not replying but then I saw your little "mysql is still shit" tag and realized you were just another idiotic troll.  I like kicking the asses of your kind, though, so this is quite enjoyable for me! 

Also, to anyone who is a fan of Postgres and isn't a flaming, retarded troll like drinkingbird, I don't think Postgres is all that bad.  It has some really awesome features that can come in handy, like domains, powerful constraint mechanisms and computed indexes.  If I was doing a small app that never needed high availability and would never use more than one database server, I'd probably use Postgres.  The extra features are nice and MySQL's advantages mostly lay in its massive scalability, speed and high availability.  So please don't think I'm saying Postgres is worthless, it's more about the right tool for the right job, like most flamewars on this board.  I only flamed drinkingbird because he was so obviously off-base and he thought it would be more fun to engage in childish name-calling and "omg Open Sauce!", anti-corporate trolling.  So don't feel the need to justify your use of Postgres to me, I'm just telling you what I have learned from working on a variety of very large, distributed systems.

morbiuswilters:

drinkingbird:

MySQL does fit the "just chuck more hardware at it" approach that Google seems to favour. Fine if that works for them.

Yeah, I hate horizontal scalability as well.  I also forgot to mention Yahoo, Facebook, LiveJournal, etc..  What large site uses Postgres?  *crickets chirping*  Oh, I guess those people just don't know how awesome Postgres is, that's it!

Postgres stopped me catching herpes! Also, ya mama's so fat, if she chucked herself at MySQL it would scale to billions of connections!

dtech:

Wow, you guys (morbius, MPS, drunken bird or something) and a local OS fan I know should really meet F2F sometime (he's the type of guy that takes his own 2,5" HD with knoppix installed on it with him because he refuses to use windows computers). I'll take my camcorder. It will do nicely as a low-bugdet apocalypse movie. :)
Only got to find a place I hate because it is going to be destroyed by a nuke, revived dinosaurs, gigantic robots or others things like that. What do you think of The White House, Washington DC? Next Friday 8 PM, can you make that?

I'll be there with bells on!

dtech:

P.S.
Morbius! You forget to add precious bytes to the reply page by not changing "__LINE__" in your tag with your fancy Greasemonkey userscript!

There's some bug with it in the last few days that causes it to not work at times.  Haven't had time to investigate it yet.  Maybe Monday when I'm at work.. 

morbiuswilters:

Honestly, my first reply with the link to MySQL was mostly meant as a joke and your response to me was so retarded I considered not replying but then I saw your little "mysql is still shit" tag and realized you were just another idiotic troll.

Yeah, maybe that tag was a bad idea. I've had nothing but shitty experiences with mysql though. From randomly corrupted tables with no discernible cause, to the mysql company distributed workbench app crashing in such a spectacular manner that it managed to drop a database in the process. Not to mention problems with ORM libraries that have used some fucked up non-standard mysql syntax, breaking compatibility with other dbs. IOW, I'm allowed to hate mysql, just as you're allowed to hate postgres.

Mysql clustering does look to be doing better than postgres. My experience in that space is limited. My initial point, however, was that mysql clustering wouldn't provide absolutely linear improvement, not that anything else would.

Calling me an idiotic troll is a bit rich, considering that you took a brief comment and completely blew up about it. You've got some serious fucking problems if you're that upset about someone else on the internet disagreeing with you. 

morbiuswilters:

What large site uses Postgres?  *crickets chirping*

Do any of these count?

emurphy:

Do any of these count?

Whoa, that list is huge!  It also doesn't say what Postgres is actually being used for on those sites.  Now compare that to MySQL's list.. 

morbiuswilters:

emurphy:

Do any of these count?

Whoa, that list is huge!  It also doesn't say what Postgres is actually being used for on those sites.  Now compare that to MySQL's list.. 

morbiuswilters:

drinkingbird:

Yeah, Postgres is definitely the "enterprise database" in this situation. The fact that it has a more permissive and straightforward licence, and isn't marketed by a for-profit company really backs that up.

Oh shit, somebody tell Oracle, Microsoft and Teradata that they make toy databases! It's true, I read it on the Internet!! Seriously, the fact that it's made by a bunch of nerds living in their parents' basements instead of a for-profit corporation makes it the more enterprise solution? I'm not even using "enterprise" saracastically, I mean large companies that need uptime and scalability.

Disclaimer: I do not state that any of the DB's is better. But I sincerely doubt that Postgres is as bad as you describe it.

Spectre:

PostgreSQL's list.

From that list:

tfa:

LAMP (Linux/Apache/Middleware(Perl,PHP,Python,Ruby)/PostgreSQL)

Ha! So THATs what LAMP means!