In today’s hyper-competitive business environment, data is one of the most valuable assets an organization can have. However, the sheer volume, variety, and velocity of data can overwhelm traditional data management solutions. Enter the data lake – a centralized repository designed to store all types of data, whether structured, semi-structured, or unstructured. Unlike traditional data warehouses, data […]
Mind the Gap: Start Modernizing Analytics by Reorienting Your Enterprise Analytics Team
… and your data warehouse / data lake / data lakehouse. A few months ago, I talked about how nearly all of our analytics architectures are stuck in the 1990s. Maybe an executive at your company read that article, and now you have a mandate to “modernize analytics.” Let’s say that they even understand that just […]
SingleStore Introduces Iceberg Integration and Faster Vector Search
According to a new press release, SingleStore has introduced several new enhancements and features aimed at revolutionizing data lakehouses and empowering intelligent applications. A highlight of their latest offerings includes a bi-directional integration with Apache Iceberg, a crucial advancement for enterprises struggling to unlock the potential of data stored in these repositories. Iceberg is renowned […]
Mind the Gap: Analytics Architecture Stuck in the 1990s
Welcome to the latest edition of Mind the Gap, a monthly column exploring practical approaches for improving data understanding and data utilization (and whatever else seems interesting enough to share). Last month, we explored the data chasm. This month, we’ll look at analytics architecture. From day one, data warehouses and their offspring – data marts, operational […]
Data Lakehouse Architecture 101
A data lakehouse, in the simplest terms, combines the best functionalities of a data lake and a data warehouse. It offers a unified platform for seamlessly integrating both structured and unstructured data, providing businesses agility, scalability, and flexibility in their data analytics processes. Unlike traditional data warehouses that rely on rigid schemas for organizing and […]
Integrating AWS Data Lake and RDS MS SQL: A Guide to Writing and Retrieving Data Securely
Writing data to an AWS data lake and retrieving it to populate an AWS RDS MS SQL database involves several AWS services and a sequence of steps for data transfer and transformation. This process leverages AWS S3 for the data lake storage, AWS Glue for ETL operations, and AWS Lambda for orchestration. Here’s a detailed […]
How to Become a Data Engineer
The work of data engineers is extremely technical. They are responsible for designing and maintaining the architecture of data systems, which incorporates concepts ranging from analytic infrastructures to data warehouses. A data engineer needs to have a solid understanding of commonly used scripting languages and is expected to support the steady evolution of improved Data Quality, […]
Fundamentals of Data Virtualization
Organizations are increasingly employing innovative technology called “data virtualization” (DV) to tackle high volumes of data from varied sources. Data virtualization is widely used in enterprise resource planning (ERP), customer relationship management (CRM), and sales force automation (SFA) systems to collect and aggregate multi-source data. From multi-sourced data acquisition to advanced analytics, this technology seems […]
Data Provisioning: Ingest, Curate, and Publish
A collection of facts from which inferences can be made is called data. It is the basis on which factual information is derived, providing relevant results to the end users. Data is the cornerstone of contemporary society and is crucial to many facets of people’s lives. In order to gain knowledge and make wise decisions, […]
Distributed Data Architecture Patterns Explained
Distributed data architecture, models using multiple platforms, and processes for data-driven goals continue to generate increased interest. As William McKnight, president of McKnight Consulting Group (MCG) and well-known data architecture advisor, says, “Seldom a database vendor does not interact with concepts around distributed data architectures: the data lakehouse, data mesh, data fabric, and data cloud, and I am […]