Data aware Data Management System.

[IDEA2]: Data aware Data Management Systems – borrowing and building up on ideas implemented in the form of OLTP, OLAP and Big Data systems and a wide variety of other systems including distributed systems (e.g. git) – developing a specification for a uniform client interface to “DATA” irrespective of its size, nature and structure.

For example providing a uniform SQL interface to data irrespective of whether the data is 1 KB, 1 GB or 1 TB and whether it is structured, semi-structured or unstructured and whether it is string, number, xml or binary content.


  1. The system should be aware of the data that it holds e.g. it should be able to adjust its physical schema and/or file system over a period of time as the size, nature and structure changes while ensuring that the client interface and the performance, scalability of the client remain unaffected.
  2. The system should be able to work in both auto-magic mode which would need minimum administration and also recommendation-engine mode which would generate and send out appropriate notifications at appropriate milestone events and allow the administrator or user to take action.
  3.  The system should follow convention over configuration yet be fully configurable.

Design Notes

  1. Cloud would be one of the important strategies to address the problem.
  2. A set of advanced tools which can monitor and scale up/down the cloud infrastructure automatically or programmatically would be required.
  3. A software platform layer can be built above these two layers (cloud and automation tools)which can automatically instruct the tools to scale up/down the  cloud infrastructure and then migrate/setup the physical schema and later choose the appropriate category of algorithms to work with the current form of physical schema. – similar to a website to capture ideas succinctly with features to share, rate, discuss. There would be only two guidelines – genuine content and freedom for everyone to use, expand, implement and/or do anything that they want to with a published idea. Patents if any that would flow from any such work need not be GPL licensed but would love if it is.