I have decided to transform all I have learned about data architecture into a method for fast implementation of Enterprise Data Governance and Management initiatives.
If you go through the current methods to do so, like DAMA and TOGAF, you will notice that they all contain the right stuff, concepts and valuable information anyone can use as a reference, BUT:
1. Propose methods and routines too complex to implement in short time periods;
2. Allow little or no flexibility in the case of subtle changes in business;
3. Do not offer fast models for enterprise governance in case of silos - most frameworks do not accept them;
4. Do not offer quick solutions to converge multiple information models into reference models;
5. Allow little flexibility in offering alternatives to standard storage models, like data warehouses and data lakes;
6. Tie data lakes to unstructured data in a mandatory fashion; the concept itself has many facets.
7. Do not propose active information models - the ones which provide self-service - in a practical way.
Based on the above, and mainly because of the lack of flexibility and excess of rules, I have decided to offer the market an alternative - one that would allow any company to quickly set up a strong and solid data management effort, ensure data quality across any silo, provide solid standards for governance and most of all, alow an innovative self-service tool, for querying existing data, create reports, export data and move data interactively across data sources.
- Polymorph is a method based on a single set of tasks, whose core is an enterprise model - a reference of all data used by the enterprise. regardless of where it came from; this reference model can become an ODS(OPERATIONAL DATA STORE) housed in a large database or a completely distributed structure.
- Once the model is defined - along with data domains, validation rules, and naming conventions, the next step refers to map all existing data sources against the reference model to map the integrations - for example, which systems map to customer, or parts or any other logical entity. During this step, duplicate sources and inconsistent data can be tracked and eventually eliminated, based on the governance rules.
- All of the products generated on the steps above are stored in a catalog that controls data structures/documentation, and executes the validation steps, suggesting and/or executing corrections.
- In the meantime, the business can decide for adopting a single environment for the ODS, or work in a distributed fashion. The architecture team makes all estimates and sizing, offering the possible scenarios. The chosen architecture becomes a new data source that receives the ODS;
- Finally, a new data instance will be defined to house the analytical environment, where all reports, KPIs, queries, and associated products will reside; that can be a data warehouse, a data lake, or a distributed instance with multiple instances, like for example those in Edge Servers, which sit between the clouds and the on-premises environment.
You will notice that this method is very flexible, provides an incremental model as a reference, and is completely independent of the solution architecture chosen, emphasizing the logical data structures but of course, mapping it against the physical environment. This approach also creates a more reliable data environment, due to "universal" standards in terms of naming, validation and format - all data must comply with some ground rules, to ensure enterprise-wide consistency.
A typical Polymorph-based architecture can be created in six to 12 months, given the complexity and size of the environment being controlled; all maintenance activities are then executed on top of finished and clearly defined models - all applications, databases and data sources, in general, will comply with the standards to ensure reliability and quality. Simplicity is at the core and costs are very low when compared to other approaches, since it aims at creating standards-related structures, regardless of their physical structures.
Silos become specialized knowledge centers, totally manageable and controlled, under standard governance rules. There is now a single knowledge base (the reference model) and all data is part of an enterprise inventory with an easy and self-serviced management platform.
If you are interested in knowing more about Polymorph, please get in touch by e-mail or phone / WhatsApp
e-mail: macr2011@gmail.com
Phone; +55 51 98325-0251
Soon I will be making some more reference documents available.
Comments
Post a Comment