Embracing Big Data Tools: Lessons Learned at EDW 2016

By on

edwby Angela Guess

Jack Vaughan reports in Search Data Management, “The onslaught of big data, with its high data volumes and diverse data structures, has given rise to new technologies in the form of NoSQL, Hadoop, Spark and the like. NoSQL, particularly, calls for changes in established data modeling techniques. Some basic learning is in order, too, when it comes to NoSQL databases, such as MongoDB, Cassandra and Redis, advised at least one data veteran during the recent Enterprise Data World (EDW) 2016 conference in San Diego. ‘Nobody is born knowing NoSQL,’ said Ted Hills, an enterprise data architect at information provider LexisNexis, based in New York. Data modelers should realize everything they know about logical modeling is still true, he continued, but they should also realize that ‘NoSQL gives a richer tool box,’ with which developers can work.”

Vaughan goes on, “Data pros should be ready to the accept change, and to embrace the new capabilities of big data tools, Hills said, even though the tools lead to changes in existing modeling methods. Hills, author of the recently released NoSQL and SQL Data Modeling, suggested a need for new modeling notations that embrace NoSQL functionality. One effect of the NoSQL side of big data development has been to delay schema creation. The early definition of the data schema was once a lynchpin of data quality practices, and a prerequisite for just getting a project going. Schema creation may be moving to a different stage in the development cycle, according to Karen Lopez, data architect and principal consultant at InfoAdvisors. ‘It’s not that we don’t care about quality. It’s that we are not caring about the schema upfront,’ she said. This doesn’t mean designs become ‘schema-less.’ Instead, they come to support something akin to ‘schema-on-read’ model, she said.”

Read more here.

Photo credit: EDW

We use technologies such as cookies to understand how you use our site and to provide a better user experience. This includes personalizing content, using analytics and improving site operations. We may share your information about your use of our site with third parties in accordance with our Privacy Policy. You can change your cookie settings as described here at any time, but parts of our site may not function correctly without them. By continuing to use our site, you agree that we can save cookies on your device, unless you have disabled cookies.
I Accept