Data Lakes are consolidated, centralized storage areas for raw, unstructured, semi-structured, and structured data, taken from multiple sources and lacking a predefined schema. Data Lakes have been created to save data that “may have value.” The value of data and the insights that can be gained from it are unknowns and can vary with the […]
Working with Complex Data Models
Complex data models have now become the norm. A single stream of the data can travel through many hubs, and many different technologies. It may travel through the front end, the APIs, the Kafka pub/sub systems, Lambda functions, ETLs, data lakes, data warehouses, and more. Riding within this stream of data is the schema, and […]
So You Want to be a Data Manager?
A Data Manager develops and governs data-oriented systems designed to meet the needs of an organization or research team. Data Management includes accessing, validating, and storing data that is needed for research and day-to-day business operations. Currently, a wide array of organizations are using Big Data to gain insights into customer behavior and to provide […]
Knowledge Graphs: Context, Compliance, and Connections
“Graph is leaving a larger and larger footprint. And that is good,” said Thomas Frisendal in Knowledge Graphs and Data Modeling. Gartner named knowledge graphs as part of an emerging trend toward digital ecosystems, showing relationships among enterprises, people, and things, and enabling seamless, dynamic connections across geographies and industries. Elisa Kendall and Deborah McGuinness, […]
The Logical Data Fabric: A Single Place for Data Integration
The ability to provide a single place for instantaneous data access can mean business continuity or closure. Many nations found this out during the recent global crisis, as countries needed to know the number of tests taken and the infection rate in order to determine both the virus’ spread and who to quarantine. Unfortunately, the […]
Data-Centric Architecture: Find Value with a Data Platform Approach
Applications provide a way to capture raw data in forms and store it in databases, and automated processes make it possible to extract meaning from that data using application programming interfaces (APIs). The current process-centric mindset assumes that the value resides in the automated processing, yet the limitations and costs inherent in its reliance on […]
Graph Databases vs. Key-Value Databases
Graph databases and key-value databases have very different features and are used for accomplishing different tasks. Key-value databases are streamlined and fast, but are limited and not as flexible. Graph databases, on the other hand, are very flexible and great for research, but not terribly fast. Both typically use a non-relational foundation. The two key […]
Case Study: Tracking and Tracing Drugs in the Pharmaceutical Supply Chain
Failures or lack of visibility in the many-tiered pharmaceutical supply chain have multiple repercussions. Drug shortages have adverse economic and clinical effects on patients — they are more likely to have increased out-of-pocket costs, rates of drug errors, and, yes, mortality. Hospitals and health systems allocate over 8.6 million hours of additional labor hours to […]
Optimizing the Data Warehouse
The data warehouse, a relational database technology, makes all enterprise information actionable, and will continue to be prominent as a Data Architecture component. In the 2000s, a typical business would consolidate data from multiple relational databases, centralizing all this information through a data warehouse, and consequently streamlining business tasks. However, the business context has shifted […]
Data Virtualization Use Cases
Data virtualization, in a nutshell, utilizes data integration without replication. In this process, a single “virtual” data layer is created to provide data services to multiple users and applications at the same time. Why Data Virtualization Is a Necessity for Enterprises explains how data virtualization helps tackle data movement challenges by making a virtual dataset […]