The trends in Data Management for 2024 can be expected to range from the impact of the EU’s Digital Services Act (DSA) package to new variations of ChatGPT focused on managing data. Data Management (DM) deals with the collection, processing, and storage of data, as well as the laws and regulations that protect people’s rights. Managing an organization’s data involves a broad range of practices, policies, and procedures.
Businesses can expect significant changes in their DM processes during 2024.
The goal of Data Management is to use data efficiently and cost-effectively while helping people to complete tasks and projects. Developing a robust DM strategy has become extremely important for organizations. A robust Data Management strategy should include an array of DM tools and techniques, and support business intelligence and analytics.
Data Management systems are traditionally developed around a DM platform, which can include software supporting databases, data warehouses, data lakes, data analytics, data integration, and more.
Changes in technology and regulations can be prepared for with the right planning. Other trends for 2024 may include:
- Automated Data Management
- The management of healthcare data
- Hybrid/multi-cloud security
The Impact of European Union’s DSA Package in 2024
The behavior and trends of businesses in 2024 will be impacted, in part, by the DSA package the European Union has developed and enacted.
The European Union (unlike the United States) has implemented additional regulations to protect their citizens: the Digital Services Act and the Digital Markets Act, also referred to as the DSA package. These acts make online activities safer and protect consumer’s and user’s rights. Enforcement will begin on March 6, 2024.
The DSA package is designed to protect the rights of users, and to level the playing field, lowering the impact of a few large platforms (Facebook, Twitter, Google, and other websites with over 45 million monthly users).
A significant concern in its development was the sale of illegal content, goods, and services online – child pornography, guns, hacking services, etc. There is also the concern that online services are being abused by manipulative algorithmic systems that are designed to amplify the spread of misinformation.
The DSA package has extraterritorial reach, and will impact businesses around the world. If an organization is doing business with European customers, even if that organization is not located in Europe, it must follow DSA rules when doing business with people or businesses within the European Union. While much of the package deals with very large online platforms, smaller businesses are impacted as well.
Smaller businesses need to be aware that the DSA package applies to all the digital services connecting European consumers to content (regarding misinformation), goods and services online (regarding illegal activities).
Organizations doing business in the EU will have to meet new obligations involving assessing and countering risks, reducing harm, protecting their users’ rights online, and meeting broader accountability and transparency responsibilities. These regulations are meant to offer new protections to internet users and make clear the legal responsibilities of organizations doing business on the internet.
Automated Data Management
Reducing the need for manual Data Management has become a key goal for certain software developers. While installing automated Data Management tools can be a complicated process, when done properly, it improves efficiency, reduces costs, and eliminates tedious manual labor. Listed below are some automated processes that organizations have started using:
- Data collection: The collection of data from different sources, such as databases, documents, and other websites.
- Data integration: This involves taking the collecting data, transforming it to an appropriate format, and storing it in a single repository.
- Data cleaning: The process of removing duplicate records, standardizing data formats, and correcting errors.
- Data processing and analysis: The use of algorithms or machine learning to develop insights from the data.
- Data Governance: This process deals with ensuring the data is handled according to the business’s policies and governmental regulations.
To keep up with the significant demands of managing huge amounts of data efficiently on a daily basis, software-based automation tools must be part of an organization’s DM practices.
In 2024, we can expect AI and ML (machine learning) to provide valuable automation services.
Maximizing Healthcare with Data Management
Unlike the banking and retail industries, the healthcare industry has not yet fully utilized data analytics or big data research. There are a variety of reasons for this lag, ranging from patient privacy to a lower emphasis on profits.
However, the healthcare industry has started using analytics and big data to find patterns. A simple example comes from France: four hospitals, all members of the Assistance Publique-Hôpitaux de Paris, used the last 10 years of their hospital admission records to make hourly and daily predictions of the number of patients they could expect at each facility. The analysis presented relevant patterns in admission rates.
Another example of data analytics in the healthcare industry is the use of real-time alerting. Hospitals have begun using Clinical Decision Support (CDS) software that analyzes medical data on the spot, providing health practitioners with advice as they make prescriptive decisions.
On November 11, 2023, the Department of Veterans Affairs entered its millionth veteran into a genetic database supporting the Million Veteran Program. The goals of their data-based research are to better understand how genes, military exposures, and lifestyle behaviors impact people’s health, and to provide individualized medicine.
Data Management for Hybrid Cloud Security
During 2024, we can expect Data Management systems to use encryption, cybersecurity mesh architecture, and network segmentation as ways to provide hybrid cloud security and protect data.
In recent years, the definition of a hybrid cloud has expanded from the combination of an on-premise system combined with a public cloud to including multi-cloud systems. The hybrid cloud supports a flexible system that provides access to specialized tools.
Unfortunately, the process of using a hybrid/multi-cloud system also comes with some security challenges.
The use of multiple clouds becomes complex from a management and security perspective. Without the proper procedures in place to track and monitor the use of various cloud’s services, management doesn’t know who is using the resources.
Additionally, they won’t know when they are being used until after they receive the bill. Because several applications use on-premises systems and multi-clouds to access and work with data, observability becomes crucial. (In this case, observability means the ability to monitor data and events across several clouds and inhouse systems.)
Vendors, such as Middleware and Datadog, have recognized this need and have focused on delivering observability tools that provide an integrated “single pane of glass” for viewing purposes.
Another concern is that different clouds use different forms of security. Developing a system that interconnects all the clouds used by your organization to work on projects presents a significant security concern, in that each connection may be a potential breach. Hybrid/multi-clouds offer significant flexibility in moving workloads between different environments quickly, but the process also increases security risks.
Data Management Using Artificial Intelligence
Although the use of artificial intelligence for Data Management purposes is not new, it does continue to grow in popularity. Prior to 2023, artificial intelligence was (and still is) used for DM tasks, acting as a more intelligent form of automated processes. Artificial intelligence is being used for a variety of DM tasks, including:
- Anomaly detection
- Metadata management
- Metadata auto-discovery
- Data cataloging
- Data mapping
- Data Governance control monitoring
With the introduction of ChatGPT, and the large language model supporting it, we can expect new solutions offering intelligent, learning-based services. As large language models continue to evolve, services supporting Data Management processes will continue to evolve with them. OpenAI, the organization responsible for developing ChatGPT, has been experimenting with Data Management.
Image used under license from Shutterstock