aaron evans

I met Tim Bray today! He’s a really nice guy and offered lots of great insight about a variety of topics.

One interesting thing that he mentioned was how innovative technologies can be derived by simplifying existing technologies. This was the case in the derivation of XML from SGML. SGML predates the rise of the Internet but SGML was not widely adopted as a general markup language for the Internet due to its complexity. XML was created as a simple profile of SGML and its simplicity caused it to become so prolific.

This immediately reminded me of an ACM article I read by former Googler Adam Bosworth. He mentioned the problems of scaling traditional databases:

1. Have databases enabled people to harness Moore’s law in parallel? This would mean that databases could scale more or less linearly to handle both the volume of the requests coming in and even the complexity. The answer is no. Things like ORDER BY, joins, subqueries, and many others make it almost impossible to push the query logic down to an arbitrary number of leaf nodes and simply sort/merge/aggregate the results. The easier way to limit queries to avoid this would be to limit all predicates to ones that can be computed against a single row at a time, at least where efficiency and scale are paramount.

As I consider all of the available alternatives to relational databases (CouchDB, SimpleDB, BigTable/AppEngine, HBase, Cassandra) I’m struck by how we could benefit from a simplified SQL dialect that would be more suitable for scaling across multiple machines (ie ‘horizontal scaling’). I’m hoping a standards body takes this on in the near future but I fear that the problem will get worse before that happens.

This entry was posted on Wednesday, July 16th, 2008 at 9:33 am and is filed under Development. You can follow any responses to this entry through the RSS 2.0 feed. Responses are currently closed, but you can trackback from your own site.