Data careers are becoming increasingly important and popular all across the globe, simply because “data” is the new currency of the data economy. The pandemic gave the needed push to accelerate the digital transformation of global businesses, and currently, the primary market differentiator is an enterprise’s data infrastructure readiness. This data infrastructure comprises systems, processes, […]
ETL vs. Data Preparation
Extract, Transform, and Load (ETL) technologies, managed exclusively by IT, have until recently been the primary tool used to combine data from multiple sources and thus provide the ability to drive important business decision making for organizations. But, with the advent of self-service data preparation, business users and subject matter experts (SMEs) can find those […]
Ten Common Issues When Using Excel for Data Operations
Click to learn more about co-author Rosaria Silipo. Using Excel to Transform/Analyze Your Data? I know you are still using Excel sheets to transform and/or analyze your data! I know, because most of us still use it to some extent. There is nothing wrong with using Excel. Excel spreadsheets are a great tool to collect […]
Going Beyond Data-Driven: The Three Pillars of Data Analytics
Click to learn more about author Eva Murray. The push for digital transformation is nothing new. Yet the accelerated adoption of digital transformation inspired by the coronavirus pandemic is unlike anything we have ever seen before. Companies have been forced to quickly ramp up their digital strategies in order to survive in our new world […]
What Is a Data Engineer?
Data engineers build Data Architecture through infrastructures and foundations. A data engineer is tasked with designing and maintaining the architecture of data systems, which incorporates concepts ranging from analytic frameworks to data warehouses. Responsibilities also include configuring, managing, and scaling data pipelines. Data engineers: Have a programming background (e.g. Java, Scala or Python) Emphasize distributed […]
Productizing IT Services for Data Management Projects
Click to learn more about author Digvijay Lamba. Many organizations overcome a lack of data engineers by outsourcing their Data Management needs to IT service providers. Startups, small and mid-sized businesses, and even larger organizations are rich in domain expertise but inexperienced in preparing data for insights, which is why they hire outsiders to perform […]
Scaling the Analytics Team: Developing Key Roles
In an enterprise analytics team, different roles exist to fill different needs, and those needs must be met in order to be successful. Launching an analytics program doesn’t necessarily require a massive influx of personnel before producing usable insights from data, yet it’s important that critical roles are filled, whatever the size of the team. […]
Data Architect vs. Data Modeler vs. Data Engineer
Michael Bowers, author and Chief Data Architect at FairCom Corporation, initially set out to research three careers in his presentation titled Data Architect vs. Data Modeler vs. Data Engineer for the DATAVERSITY® Data Architecture Online 2019 Conference. The process brought him to a wealth of information he would have appreciated much earlier in his career, […]
Data Architecture and Data Science: What is the Intersection?
Data Science, in practice, should ultimately combine the best practices of information technology, analytics, and business. On the other hand, Data Architecture enables data scientists to analyze and share data throughout the enterprise for strategic decision-making. Thus, without a sound Data Architecture in place, data scientists will remain severely handicapped in their abilities to develop […]
Case Study: Deriving Spark Encoders and Schemas Using Implicits
Click to learn more about author Dávid Szakallas. In recent years, the size and complexity of our Identity Graph, a data lake containing identity information about people and businesses around the world, begged the addition of Big Data technologies in the ingestion process. We used Apache Pig initially, and then migrated to Apache Spark a […]