Implementing Data Fabric, Mesh, or Lakehouse

Data Mesh has emerged in the past year or so as one of the most talked about topics in analytics. Data Lakehouse has captured the attention of many organisations struggling with Data Lakes. Meanwhile, Data Fabric is being proposed by many vendors and analysts as the new way forward for Data Warehouses. All three patterns propose novel, varied, and partially overlapping solutions to old data delivery problems.

But what are they? Are they truly novel or simply marketing hype? Are they the same thing? How do they relate to the Data Warehouse, Lake, or Hub? What are their benefits and drawbacks? Should you be planning a Mesh or a Fabric? Is a Lakehouse the solution you need? Whichever you choose, where would you start?

Data Mesh, the most unusual of the three patterns, suggests you learn from agile, domain-centric software development and eliminate centralised bottlenecks, such as enterprise Data Warehouses. Data Lakehouse proposes you start from your Data Lake and build a Warehouse on it. Data Fabric starts from metadata and recommends automating your data delivery infrastructure.

However different, all these new and old terms, with overlapping scopes and diverse promoters, are types of digital information systems, designed to manage and deliver data/information to all digital business processes in today’s complex distributed and network-centric environments. In this seminar, Dr. Barry Devlin explains and positions Data Fabric, Lakehouse, and Mesh, as well as other concepts, old and new, using as a foundation the Digital Information Systems Architecture (DISA) first defined in “Business unintelligence.”

Building on the conceptual and logical architectures, we examine how implementation differs between the three patterns, what are the starting points, and what the principal challenges are.
Existing and emerging technologies for data storage, preparation, and virtualization; data catalogs; and other tools, both on-premises and Cloud, are described. Also explored are a variety of organisational issues, methodologies, and implementation approaches.