Implementing Data Fabric, Mesh, or Lakehouse
Unravelling Digital Information Systems
Data Mesh has emerged in the past year or so as one of the most talked about topics in analytics. Data Lakehouse has captured the attention of many organisations struggling with Data Lakes. Meanwhile, Data Fabric is being proposed by many vendors and analysts as the new way forward for Data Warehouses. All three patterns propose novel, varied, and partially overlapping solutions to old data delivery problems.
But what are they? Are they truly novel or simply marketing hype? Are they the same thing? How do they relate to the Data Warehouse, Lake, or Hub? What are their benefits and drawbacks? Should you be planning a Mesh or a Fabric? Is a Lakehouse the solution you need? Whichever you choose, where would you start?
Data Mesh, the most unusual of the three patterns, suggests you learn from agile, domain-centric software development and eliminate centralised bottlenecks, such as enterprise Data Warehouses. Data Lakehouse proposes you start from your Data Lake and build a Warehouse on it. Data Fabric starts from metadata and recommends automating your data delivery infrastructure.
However different, all these new and old terms, with overlapping scopes and diverse promoters, are types of digital information systems, designed to manage and deliver data/information to all digital business processes in today’s complex distributed and network-centric environments. In this seminar, Dr. Barry Devlin explains and positions Data Fabric, Lakehouse, and Mesh, as well as other concepts, old and new, using as a foundation the Digital Information Systems Architecture (DISA) first defined in “Business unintelligence.”
Building on the conceptual and logical architectures, we examine how implementation differs between the three patterns, what are the starting points, and what the principal challenges are.
Existing and emerging technologies for data storage, preparation, and virtualization; data catalogs; and other tools, both on-premises and Cloud, are described. Also explored are a variety of organisational issues, methodologies, and implementation approaches.
What you will learn
- History, meaning, and detailed functionality of Data Fabric, Data Mesh, and Data Lakehouse
- An introduction to the Digital Information Systems Architecture (DISA) and its business and technical uses
- Technical rationale, structure, and components of the DISA conceptual and logical architectures
- An in-depth comparison of Data Fabric, Mesh, and Lakehouse with Data Warehouse, lake, and Hub using DISA as a basis
- Possibilities and challenges of new database and data management technologies in Cloud, on-premises, and hybrid environments
- The central role of context-setting information and metadata
- Adaptive Processes as the basis for data preparation, information creation, and insight discovery
- Using data virtualization and preparation as tools for integration of all types of content and data in Cloud, on-premises, and hybrid environments
- Practical planning and implementation steps from Data Warehouse/Lake to Data Lakehouse, Fabric, or Mesh
- Introduction to digital information systems and their evolution
- Digital Information Systems Architecture
- Deep dives: Data Lakehouse, Fabric, and Meshù
- Shallow dives: Data Warehouse, Lake, Hub etc.
- Information storage/management (including metadata) technology review
- Information preparation and distribution technology review
- Planning and implementation considerations