The Data Aware Platform
Unsure of the right architecture to meet your applications needs. Why not let your application store tell you how your applications are actually using the entire ecosystem (compute, storage, and supporting infrastructure).
Data Management I/O (DMIO)
For DMIO v1, the engine interprets a custom user-defined application context, analyzes the object data structure to be saved, and routes the data,using the context and I/O access-patterns of the application, to an underlying highly-performant and scalable data store.
Depending on the applications context, we will route data to ensure that the correct ACID semantics, caching needs, and data durability needs are met.
Using our Smart Store and MetaData Engine Subsystems, we track all movement through the whole LDI ecosystem. Those being telemetry data, data access patterns, data-structure, all in real-time. We use these 3 subsystems to build recommendations for how we manage the data. These subsystems also provide the ability to provide recommendations back to the application about the system is actually being used. Using system metrics is just one way that we are are building a data and application aware system.
How does this currently work? Under the hood we use proven Open Source Apache technologies: MariaDB, Lucene-oriented document indexing, HBase, etc. We use these data stores in conjunction with our MetaData and Smart Store Engines to build the right model for how data is routed, stored, and consumed.
Data Aware Platform and the LDI Mindset for our Next Gen Product
The concept is simple. The implementation is really, really difficult.
Today, most applications create objects that structure data logically and immediately persist the data to some data-store. Then, that object is reincarnated on some later date when someone or something needs to read or update the object's properties. This serialization/de-serialization pattern can happen many, many times in life-cycle of the object. So, there's all this code to govern how this object is structured and what it can do. But, why not keep that object in an executable state, track its mutations for downstream consumers, or provide a real-time view of the data for advanced analytics.
To accomplish this, we have to stretch both, what we know and how we create most applications and data today. But, what if, your objects acted like a process. That is, they had access to CPU and memory and weren't destroyed as fast as they were created. Then, they could provide a new dimension of access for users, applications, external entities. If you had a scalable platform for runtime, you could then code your applications to take advantage of being in a constant running state. You could provide new avenues of access for advanced analytics, new ways of generating insights in real-time, react faster to real-time triggers, disruptions, and events in the application's ecosystem. our Living Data Infrastructure is the base foundation for building such a platform.
Want further information...
For further information on LDI or DMIO, please email our Product Development team at engineering@somethingdata.com.