Loading...
You are here:  Home  >  Data Education  >  Big Data News, Articles, & Education  >  Big Data Articles  >  Current Article

Enhanced Real-Time Data Processing with Redis Labs’ In-Memory Database Solution

By   /  July 5, 2017  /  No Comments

in-memory databaseIn 2009, Salvatore Sanfilippo began the Redis project, a supplemental software program designed to improve LLOOGG (currently an open source project), by helping it to process data more quickly and efficiently. From there, the Redis project blossomed into Redis Labs, providing services and products to augment a customer’s Data Management potential through an advanced, NoSQL, highly available, scalable, in-memory database platform – Redis Enterprise.

Three strengths offered by the Redis Enterprise which powers Redise Cloud( a Cloud service) and Redise Pack(downloadable software) are speed, persistence (saving your info), and the availability of a variety of data types.

In essence, Redis is a “very fast” NoSQL, in-memory database that supports numerous platforms, and is used as a cache, a data store, and a message broker. It allows the user to access a highly scalable data store offering multiple processes, multiple applications, and multiple servers. Redis’ Key-value store uses an associative array (also referred to as a dictionary or a map) as their base data model. This simple model is fast, flexible, scalable, and easy to use due to its advanced in-memory database architecture.

Redis has expanded to include many data structures. These data structures resolve highly complex programming problems using simple commands that are processed within the data store. This has the effect of increasing throughput, reducing latency, and reducing coding efforts. Recently, Redis added “modules”, a compact way to add custom data structures to Redis – which can be used to add multiple types of data processing functionality like graph, document, JSON, search and can even include processing that traditional databases don’t typically include – for example, image processing or machine learning.

According to Leena Joshi, Redis Labs VP of Product Marketing:

“Redis includes these data like lists, sets, sorted sets, hashes, bitmaps, pub/sub, and with these data structures, what you can achieve is a smorgasbord of functionality, all from one database with built-in analytic commands.”

She gave the example of a plain, minimal website that is implemented in a way where the user comes to the website, all the data the user is clicking on is stored and can be viewed and analyzed later. “That’s your basic implementation example,” she said.

“Where the market is going now, as the user is doing things on your website, you want to present him with a dynamic view. These are the items you should click on, or, these are the things we think you will like, because you belong to this segment, and we’ve already profiled you, and other people who are like you, like these things.

Now, it’s necessary to execute that in real time. A business needs something that can retrieve information in the moment of ingesting it, as well as computation and analytics in real time. They need to be able to do intersections, and “incorporate Machine Learning models. And, that’s where we see Redis really taking off,” remarked Joshi.

The Redis Cloud

Persistence can be quite valuable when working in the Cloud. Consider the frustration of an unexpected restart and the loss of all information and hours of labor. Not so with Redis. It can be used as a true database instead of just a volatile cache (assuming you want this feature). Redis offers, not one, but two options for persistence, Snapshot (per database) and Append Only File (AOF). These two systems are used specifically to restore a database, should it fail. (Data persistence at Redis is an optional feature and “can be” set to NONE).

The variety of data types is also remarkably useful. Key-values can be made up of simple strings, and more complicated associations such as Hashes, Lists, or Sorted Sets. Additionally, Redis supports other useful components, such as their built-in pub/sub and Lua scripting. Modules that add functionality to core Redis include RediSearch, Neural-Redis, Redis graph, ReJSON and many more.

The Redis Cloud offers a public, less secure version, and a private, more secure version. Cloud environments available to work with are:

  • Amazon Web Services
  • Microsoft Azure
  • Google Cloud Platform
  • IBM SoftLayer
  • Heroku
  • Pivotal Web Services
  • OpenShift
  • other PaaS providers

Because the Redis Cloud offers an extremely scalable data store with multiple processes, multiple servers, and multiple applications, it can be used for inter-process communications. Estevan Seneca, a writer for Quora.com, wrote:

“I use Redis as a pub/sub and Task Queue for my app. I find it to be full featured providing opportunity to scale well. You can fine grain controls such as how the disposition of different servers appears to one another (i.e. Master/Slave); as well as how to treat stale data. Using the various types that Redis supports you can further optimize access times and throughput by tailoring how you store information for the most rapid sorting/fetching operations.”

The Redise Pack

The Redise Pack (and the Redise Pack Managed, which comes with live support) is downloadable software that uses a clustering technology, which, in turn, provides several benefits. The Redise Pack can be installed in the computer environment of choice. This can be an on-site data center or the Cloud platform of preference. “The program’s architecture supports a variety of databases created by different users, while simultaneously keeping all databases isolated from each other,” said Joshi. Additionally, the program provides the flexibility needed to run multiple configurations, allowing Redis to offer “true” high availability.

High availability is the ability to replicate a database within the same data center, and to replicate and store a dataset across other data centers and regions. This provides seamless recovery and certain performance benefits. Should a node fail, the data is automatically found, and sent from a replacement node within the cluster. According to Joshi, Redis technology automatically handles node failures, programming failures, and proxy failures. “Auto-failover” takes place within a few seconds.

The Redise Pack does require a little conscious forethought before purchase and installation. It is remarkably fast because it stores all (or most) data and changes, directly to RAM. This means it needs to be installed in a server with a great deal of primary memory available to maximize the program’s full potential. Redise Pack also includes Redis Enterprise Flash technology which allows SSDs to be used as an extension of RAM. This allows for data to be distributed in a tiered manner (“hot” data in RAM, “cold” in Flash) without significant impact to latencies, which still stay in the sub-millisecond range while reducing costs – since Flash is ten times cheaper than RAM. This fact should be included when weighing the overall costs of installing the Redis Pack.

The Redise Pack enables architecture that supports multiple databases developed by multiple users over the same cluster infrastructure, and simultaneously keeps the databases completely isolated from each other. It also provides the flexibility needed in supporting a customer’s specific performance and availability goals. Each Redis database can individually conform to one of the following templates:

  • Highly-available cluster (multiple master/slave shards)
  • Cluster (multiple master shards)
  • Highly-available (a single master/a single slave)
  • Standalone (a single master/Redis server)
  • Redis Flash

Redis Flash

The base technology of Redis Flash operates Flash as an extension of RAM, rather than persistent storage. All ‘hot’ values, and the global key list, are stored in RAM. All ‘cold’ values (typically those making up the bulk of the dataset) are kept in Flash.

Redis Flash is good fit for scenarios where values are noticeably larger than the keys, and “hot” objects make up 10-40 percent of all stored objects. Compared to a total RAM deployment, Redis Flash can save as much as 70 percent in resource costs.

Leena Joshi stated during the interview that:

“Since we’ve expanded the pool with flash memory, which is 10 times cheaper than regular memory, you can think about much larger data sets, such as analytics data sets, and this is pretty significant. We have, in Redis Flash a tiered access, i.e. a second level of tiering inside Redis, which are keys and hot values stored in RAM, and cold values are stored in Flash, which means it is like another cache inside your Redis deployment.”

Awards and Accolades

The Forrester Wave provides an overview of various leading service providers in the world of business and technology. In the 2017 report, Redis Labs was given the title, “strong performer,” based on receiving the highest attainable evaluation scores. Redis Labs has received a variety of other awards as well, which are listed on its Industry Recognition webpage.

 

Photo Credit: nikko tee/Shutterstock.com

About the author

Keith is a freelance researcher and writer.He has traveled extensively and is a military veteran. His background is physics, and business with an emphasis on Data Science. He gave up his car, preferring to bicycle and use public transport. Keith enjoys yoga, mini adventures, spirituality, and chocolate ice cream.

You might also like...

The Data Governance Playbook: Sixteen Steps to Better Data Privacy

Read More →