by Angela Guess

The 2012 NoSQL Now! Conference featured a number of excellent discussions surrounding NoSQL solutions, advancements, tools, and use cases. One such discussion was led by Josiah Carlson, Chief Architect of ChowNow Inc. Carlson’s presentation was titled Why and When You Should Use Redis. Redis is an in-memory data structure server that has grown in popularity over the last few years for a variety of reasons. Being faster than the majority of other available databases (over 300k ops/second have been measured on 4-year old hardware), offering a data model unique to Redis, and support for master/slave replication are just a few. From the author of Redis in Action by Manning Publications, this talk introduces what Redis is, what it does, and the types of operations it supports.

At ChowNow, Josiah Carlson builds and maintains an online ordering platform for restaurants. On the evenings and weekends when not spending time with his wife, Josiah works on Redis in Action, which has 6 of 12 chapters written and is available to read now. In the past, Josiah built a Twitter analytics platform and ad serving network for Ad.ly, where he also served as Chief Architect prior to ChowNow. Josiah also spent over a year and a half at YouTube/Google, helping with the home page, channels, and groups products. While finishing his Ph.D. in Computer Science Theory at UC Irvine, Josiah worked at Networks in Motion (now Telecommunication Systems) as a Senior Software Engineer, where he helped with the GPS navigation product, helping to ship the first cell phone GPS navigation system to include traffic-enabled navigation and traffic updates while driving.

Derrick Harris of GigaOM reports, ” It doesn’t have a cool name like Cassandra, Voldemort or MongoDB, but Google is offering up a non-relational database called Google Cloud Datastore. Like almost everything the company has done since announcing its Compute Engine service at last year’s IO conference — including the rest of the features it announced on Wednesday — Cloud Datastore looks like a direct shot at current cloud champion Amazon Web Services. AWS has a managed NoSQL database service called DynamoDB that’s replicated across three availability zones to ensure its stays up. Google’s Cloud Datastore sounds eerily similar, according to the product’s website (although Google calls its product “NoSQL-like). It’s fully managed, built for speed and scale and is replicated across data centers. For some queries, Google even promises that Cloud Datastore will support ACID transactions.

Harris continues, “Although the services advertise similar features in terms of availability and scalability, they’re quite different technically. Cloud Datastore is based on Google’s BigTable database (and a library called Megastore on top of it) while DynamoDB is based on Amazon’s Dynamo database. You can get details on Datastore  and how it works here. Pricing information is available here. If its goal is to compete with AWS, though, Google’s cloud platform still has a long way to go. Yes, it has most of the key services in place and even some seeming advantages in certain areas, but it’s lacking the incredible breadth of services AWS offers — everything from virtual server instances to a devops service to a hosted data warehouse. It’s also lacking a seven-year reputation for being an all-around reliable platform and an ever-growing list of large-enterprise users.”

Read more here.

photo credit: Google

Doug Henschen of Information Week reports, “Developing an integrated customer view has been on the wish list at insurance giant MetLife for at least 10 years, but it recently took a fresh approach to the challenge by choosing a NoSQL database as the platform for bringing together data from more than 70 separate administrative systems, claims systems and other data sources. It moved from pilot to rollout in 90 days — breakneck speed in an industry used to measuring IT projects in months and years. ‘We had 60 different teams working together as one group, and they were working nights and weekends not because they had to but because they were excited and wanted to,’ says Gary Hoberman, MetLife’s senior VP and CIO of regional application development.”

He continues, “Ripping out, replacing or otherwise touching these mission-critical systems of record was out of the question. So how could MetLife access information from these diverse sources? NoSQL databases have emerged in recent years as a diverse and scalable option. MetLife also confronts its share of data-quality and data-diversity challenges within systems. By definition, life insurance and annuity products are long lived, but as healthcare and the insurance business have evolved, so, too, have data-collection requirements and standards. Today’s policy records, for example, have many more fields of data than the records behind policies issued in the 1990s, 1970s or 1950s. Look across the corpus and you have what might be described as ragged or sparse data with missing fields — another argument for NoSQL. Finally, MetLife deals with semi-structured and unstructured information, such as images of health records and death certificates. This contributed to MetLife’s selection of MongoDB — 10Gen’s open source document database — over other NoSQL alternatives such as Cassandra, which MetLife is testing in other applications.”

Read more here.

photo credit: MetLife

The document database, MongoDB, is currently one of the most prevalent NoSQL databases on the market. MongoDB offers both open-source and enterprise versions with the complete support of its developers 10gen. It provides an innovative tool for an individual developer experimenting with the freely available community edition and more robust functionality for a large corporation looking for additional flexibility and scalability than what is found with most relational databases.

10gen began development of MongoDB in 2007 and the database entered production ready status three years later. 10gen found their original inspiration in trying to create an “as a Service” product similar to Google’s App Engine, or Microsoft’s Cloud computing service, Azure. Seeing how relational databases could not support the requirements of the modern Web for scalability and wide distribution, the company began developing a database to meet these needs, eventually discontinuing work on the Cloud services platform after open sourcing MongoDB in 2009.

MongoDB is currently in use at a wide array of organizations, both in the public and private sector. Forbes, Craigslist, MTV Networks, The National Archives, Disney Interactive Media Group, Cisco, and Reverb Technologies collectively make up a portion of MongoDB’s user base. The name, MongoDB, stands for “huMONGOus” database.

MongoDB Basic Features

MongoDB offers the same capabilities at the database level in both editions, including the free community version. For larger companies looking for enterprise-ready features and support, 10gen provides commercial licenses that include those features, noted later in this article.

Documents marked up using a binary formatted version of JSON (JavaScript Object Notation) are the basic storage element in a MongoDB database instance. Flexible schemas are fully supported, so successive documents in the same collection don’t have to share a common structure. Even if they do hold a common structure, the same fields in successive documents can each store a different data type.

This data modeling flexibility allows developers and modelers to closely align a database structure with an application’s object models. The documents are essentially programming objects, after all. Indexes, as well as normalization and de-normalization are possible, allowing fine-tuning of the performance of any application using MongoDB as its persistence engine.

MongoDB easily scales both horizontally and vertically, with full support for sharding and map/reduce processing. Mirroring across LANs and WANs, in addition to MongoDB’s replication functionality, combine to help ensure a high availability factor.

As mentioned before, documents in MongoDB are stored in BSON, which is a binary representation of JSON. Since BSON documents are limited in size to 16MB, MongoDB supports GridFS, allowing larger documents to be successfully managed by the database. GridFS breaks up larger files into chunks, with one collection storing the chunks, while another collection stores the file’s metadata.

MongoDB provides a command line shell that fully supports the JavaScript language. In fact, all database CRUD operations are available through JavaScript commands like find, update, save, insert, upsert, and remove. A wide range of client drivers are also provided, supporting the following languages: Python, Ruby, PHP, Perl, Java, Scala, C#, C, C++, Haskell, and Erlang. The C# driver includes support for the .NET Framework’s LINQ (Language Integrated Query) feature.

Getting Started with MongoDB

The open-source version of MongoDB is freely available for download at MongoDB.org. Versions exist for Mac OSX (64-bit), Linux (32 and 64-bit), Windows (32 and 64-bit), and Solaris. Additionally, a demo version of MongoDB Enterprise is available directly from 10gen.

Before experimenting with MongoDB, it is necessary to know how to use a command line shell as well as being at least somewhat familiar with JavaScript syntax. A host of documentation and tutorials are available for MongoDB, but the database install doesn’t include a graphical IDE for administration. The mongo shell provides a full JavaScript environment with access to the standard database functions; it assumes a running database server available through a localhost interface.

For users who just want to try out the shell without dealing with setting up a database server, 10gen provides a useful browser-based version at MongoDB.org. It automatically connects to a database and includes a basic tutorial that covers all the basic database CRUD functions.

The convenient MongoDB browser-based shell steps a user through creating, reading, updating, and deleting a few document database records.

When running the tutorial, the user creates a few document records and adds them to a database collection using JavaScript functions. The tutorial then explains how to query those records using the find statement, in addition to the other CRUD functionality. While running the tutorial, this handy SQL to MongoDB conversion chart serves nicely to illustrate the differences between the two databases, especially when it comes to syntax.

Querying the new “artists” collection for jazz and rock albums using the MongoDB browser shell.

The MongoDB browser shell and its embedded tutorial are a great way to wet the appetite and get a feel for this version of the CRUD process before diving into a full MongoDB install and setting up a database server. Installation instructions for MongoDB are available here for each platform.

Enterprise Ready Commercial MongoDB Licenses and Support Options

As mentioned earlier, larger companies needing a range of enterprise-ready features for their MongoDB application have a host of options provided by 10gen. A commercially-licensed version of MongoDB, branded as MongoDB Enterprise, includes Kerberos Authentication which allows easy integration into existing security systems.

On-premise monitoring is another feature of Enterprise, offering a collection of over 100 system metrics. It works in a similar fashion as 10gen’s Cloud-based monitoring service which is freely available to all MongoDB users, regardless of license. Users with support subscriptions gain the additional benefit of having 10gen’s engineers predictively analyze deployment issues and make recommendations before problems occur.

MongoDB Enterprise also offers support for SNMP – Simple Network Management Protocol – which facilitates the integration of the database into other enterprise applications and monitoring services.  Additionally, Enterprise is certified to operate with a variety of Linux distributions, providing additional piece of mind for larger corporate installations.

While the Enterprise version of MongoDB includes some additional functionality suitable for an enterprise deployment, the open source edition essentially works the same at the database level. 10gen offers three levels of MongoDB support subscriptions, with the highest level including access to MongoDB Enterprise. Additional differences between support levels, other than the per server cost, relate to the Service Level Agreement, support availability, and whether or not emergency software patches are included.

Finally, 10gen recently introduced into limited release a Cloud-based backup service for MongoDB instances. It follows a “pay as you use” billing model, with an agent process running on the server performing backups in the background every 6 hours. It retains multiple copies based on a set retention policy, with restores performed by 10gen on demand.

Options for Learning More about MongoDB

The MongoDB.org website includes the documentation, installation guides, and tutorials that serve nicely in getting anyone started using the database. There are separate sections suitable for developers and administrators as well as a full reference covering shell methods, in addition to database commands and query operators. For a higher level view, 10gen provides a collection of case studies, datasheets, and white papers on their website.

Considering the rapidly growing popularity of MongoDB, it makes sense that a large collection of books have hit the stores in the past year or so. MongoDB: The Definitive Guide from O’Reilly Books is recommended by 10gen; it looks to be a robust volume that provides a full overview of the database, including administrative functions, in addition to client development examples for the Java, PHP, Python, and Ruby languages.

The NoSQL movement continues to increase in relevance as society becomes ever more interconnected with a resultant exponential increase in data volume. MongoDB is one of the major databases at the center of this movement. It is a technology worth exploring for every database professional.

Don Pinto of Couchbase writes, “The bottom line for any sysadmin is to keep the database and in case of Couchbase, keeping the cluster up and running 24×7. With demanding app requirements, your cluster needs to be properly configured, sized and monitored. This can be quite challenging as things can fail anytime without any prior warning. As an operator, everyday is different – full of surprises and challenges. But, there are a few things every Couchbase Server operator must know. We hope that the following 10 things will come in handy and make your job easier.”

First on the list: “Keep your client libraries up-to-date. Always use the latest version of the client libraries compatible with the server. By using the latest client libraries, you can get the most out of your Couchbase cluster – recently tested code and the latest features surfaced through the client.” Second: “Monitor, monitor and monitor. By using the admin dashboard or the REST API’s, you can monitor how your Couchbase cluster is doing. It is a good practice to monitor the following system metrics – cache hit ratio, disk reads, resident item ratio and disk write queue.”

Read more here.

 To view just the slides from this presentation, click HERE.

This Webinar was Sponsored by:

10gen | the MongoDB company

About the Webinar

Since relational databases were introduced in the 1970s, the environment has changed dramatically, and they are no longer best suited to solve all the challenges of today’s environment. Data types have evolved and volumes have grown; application development now occurs over weeks or months, not quarters or years; and ops teams are leveraging commodity and cloud architectures across multiple data centers, instead of single monolithic deployments. The NoSQL movement has come about to address the shortcomings of relational databases and the demands of modern software development, and it’s quickly gaining mainstream attention. In this webinar, learn about NoSQL database and how they can help your business address its challenges more quickly and efficiently.

About the Speaker

Graham Neray, Product Marketing Manager, 10gen

Graham is a Product Marketing Manager at 10gen, supporting initiatives in product marketing and alliances. Prior to joining 10gen, he was a Senior Business Analyst at CSMG, a boutique management consulting firm specializing in the high tech and telecom industries. He graduated from Brown University with a concentration in Political Science.

To view the recording of this webinar, click HERE.

This Webinar was Sponsored by:

10gen | the MongoDB company

About the Webinar

Since relational databases were introduced in the 1970s, the environment has changed dramatically, and they are no longer best suited to solve all the challenges of today’s environment. Data types have evolved and volumes have grown; application development now occurs over weeks or months, not quarters or years; and ops teams are leveraging commodity and cloud architectures across multiple data centers, instead of single monolithic deployments. The NoSQL movement has come about to address the shortcomings of relational databases and the demands of modern software development, and it’s quickly gaining mainstream attention. In this webinar, learn about NoSQL database and how they can help your business address its challenges more quickly and efficiently.

About the Speaker

Graham Neray, Product Marketing Manager, 10gen

Graham is a Product Marketing Manager at 10gen, supporting initiatives in product marketing and alliances. Prior to joining 10gen, he was a Senior Business Analyst at CSMG, a boutique management consulting firm specializing in the high tech and telecom industries. He graduated from Brown University with a concentration in Political Science.

John Rath of Data Center Knowledge reports, “MapR Technologies has announced an update of its Big Data platform that provides performance improvements for NoSQL and Hadoop applications. With the MapR M7 Edition, MapR says it has removed the trade-offs organizations face when looking to deploy a NoSQL solution. M7 delivers over one million operations per second with a 10-node cluster, and can support up to one trillion tables across thousands of nodes, according to MapR. It will perform automatic region splits and self-tuning with no downtime required for any operation, including schema changes. The MapR M7 edition is available immediately.”

Rath quotes John Webster, principal analyst at Evaluator Group, saying, “The number of enterprise-level deployments of Hadoop MapReduce is rising quickly, driven by a need to understand and potentially adopt this new business analytics platform for business applications… Responding to this demand, MapR delivers a distribution of Apache Hadoop that addresses many of the enterprise quality issues currently limiting its adoption in production data centers. With M7, HBase applications can access data directly without the redundancy of extra layers of communication yielding a single, scalable and more reliable data store that offers high performance and is easier to develop to and administer.”

Read more here.

photo credit: MapR

According to a new article out of the company, “Precog, the leading vendor for NoSQL analytics, today announced the general availability of its analytics platform for non-relational data. The announcement comes approximately 6 months after Precog launched its public beta. With the GA release, Precog has formalized pricing for its cloud offering, which starts at $500 on the low-end (for storing and analyzing up to 10GB on SSDs), and reaches $5000 on the high-end (for up to 500 GB). In addition, the company has launched two new deployment options, including a virtual appliance and a hardware appliance, both designed with the needs of Enterprise customers, and which range in price from $5000 to $60000.”

The article continues, “According to company founder and CEO John A. De Goes, more than 1,300 companies have signed up for beta accounts since October. De Goes states, ‘Our rapid growth is fueled by the explosion of non-relational data. To use legacy analytics solutions on NoSQL data, you have to first make the data relational through costly, time-consuming, fragile transformation processes. Companies are fed up with this incessant pounding of a square peg into a round hole.’ Continued De Goes, ‘Even when you normalize your non-relational data, you can’t do much with something like a tweet inside a relational database except count it. Companies are moving beyond sums and counts. They want to ask deeper questions across all of their data, whether it’s well-structured tabular data, or messy unstructured data’.”

Read more here.

photo credit: Precog

Andrew Lampitt of InfoWorld reports, “Netflix is the big Kahuna of a Web media businesses, with 33 million subscribers in more than 40 countries. As Netflix’s “watch now” streaming service has grown, the company has had to rethink its data and storage strategies to cope with ballooning workloads managed in the cloud. Today, the company is nearly complete in its migration from Oracle to the NoSQL database Cassandra, improving availability and essentially eliminating downtime incurred by database schema changes.”

He continues, “Netflix launched its streaming service in 2007, using the Oracle database as the back end. ‘We had a single data center, which meant we had a single point of failure,’ explains Adrian Cockcroft, cloud architect at Netflix. ‘We were approaching limits on traffic and capacity. Now that people can watch Netflix streaming programming from their phones, from Wii devices, Roku boxes, and many others, the demand for availability increases all the time. We have more customers every quarter, more customers are using streaming, and they’re using streaming at a greater rate.’ Data has grown as fast as the customer base, Cockcroft says: The number of API requests in January 2011 was 37 times higher than requests in January 2010. The company knew that outages or poor-quality streaming could drive away customers. ‘We knew we had to get out of the data center, so we could keep running and keep growing,’ Cockcroft says.”

Read more here.

photo credit: Netflix