SmartData Collective continued their series on root causes of poor data quality with two more problems. The first is corporate evolution: “An organizations undergoes business process change to improve itself. Good, right? Prime examples include: company expansion into new markets; new partnership deals; new regulatory reporting laws; financial reporting to a parent company; downsizing.  If data quality is defined as “fitness for purpose,” what happens when the purpose changes? It’s these new data uses that bring about changes in perceived level of data quality even though underlying data is the same. It’s natural for data to change. As it does, the data quality rules, business rules and data integration layers must also change.”

Another root problem is “secret code.” The article explains, “Databases rarely start begin their life empty. The starting point is typically a data conversion from some previously existing data source. The problem is that while the data may work perfectly well in the source application, it may fail in the target. It’s difficult to see all the custom code and special processes that happen beneath the data unless you profile.”

Read about how to address these problems here.

photo credit: psyberartist

Rob Armstrong recently reminded data professionals that Big Data isn’t about the ‘big,’ it’s about the data. He writes, “It is a little amusing to those of us that have been in the data warehousing game a long time, especially with Teradata, to hear the term BIG DATA. At the start of Teradata, people openly wondered if anyone would ever have a need to handle a terabyte of data, and yet today there is talk of petabytes and beyond with hardly a blink of an eye.”

Armstrong continues, “Big data is not about volume, though it may certainly be voluminous. The accent is really on the second part, the data. The term applies to many new and varied data sources such as RFID, Geospatial, Smart meters, social network data, and web logs. Yes this data can be big but more importantly it provides a source of very rich analytical opportunity.”

He adds, “You can think of this is a few ways. There is data such as smart metering for a utility company. Getting a discrete usage tracking at 15 minute intervals will certainly cause much more data coming into the environment. As opposed to a once a month recording this would be approximately a 3000 times jump (96 a day for 30 days rather than a 1 row per month)! But the data is still the same structure and very “relationally friendly”. The data fits the existing model and can enable much better analytics that leads to network optimization, marketing new rates and services, and integration into home appliances. So here the data is big but easy to handle, provide you have the technology and scalable environment to allow that type of growth.”

Read more here.

photo credit: CharlieBaxterPhotography.com

Forrester Research has released a new report (available for a fee) entitled Master Data Management: Customer Maturity Takes a Great Leap Forward. According to one article, the report describes how mystery is morphing into best practices as MDM begins to mature. The Forrester report “categorized the basis and depth of about 175 inquiries from enterprises and information managers to the analyst firm on MDM plans January 2010 to June 2011. Forrester found the MDM focus for enterprises shifting to best practices (27 percent), governance and ownership (24 percent) and strategy (16 percent). Questions on vendor selection came in at 22 percent, about 20-percent less than MDM inquiries from three years ago, says Rob Karel, Forrester analyst and author of the report.”

Karel writes in the report, “Back then, most data management pros were still figuring out what MDM was and deciding if they should care, so it’s crazy that almost half were already talking vendors before they even built their business case or organizational strategy… I’m confident that vendor conversations now taking place are a lot more productive and, from a vendor standpoint, qualified than two-to-three years ago.”

The article adds, “In all, MDM-related inquires have increased 90 percent since 2008, Forrester found. Forrester has estimated the MDM market to reach a new high of $2.2 billion in software sales for 2011. Implementation is still somewhat in the early stages, as about one-third of requests on best practices related to getting started, and only 15 percent centered on improving existing programs. But as interest and planning mature, Karel anticipates implementation, strategy and fine-tuning of data oversight to develop along the same lines over the next few years.”

photo credit: Alex E. Proimos

Sports Authority is searching for a Director of Information Management in Englewood, CO. The post states, “This position will be responsible for implementation, development, and support of systems related to Business Intelligence, reporting, ETL, data modeling, intranet portal, and other systems as determined by management. This position will also manage EDI operations and provide oversight of vendor relationships related to enterprise systems software development and support.”

Responsibilities of the position include: “Provides leadership and oversight to all systems development activities for assigned department. Ensures strategic alignment and prioritization of systems development with overall IS and corporate strategies. Provides Leadership on design and support activities related to assigned departments. Ensure high levels of customer service and production support for the business and other IS departments. Identify root causes of problems and provide resolution to eliminate recurrence. Lead, oversee, and provide support to assigned team managers and group to ensure compliance and continuous improvement.”

Learn more and apply here.

photo credit: Sports Authority

Evans & Chambers Technology is looking for a Big Data Technical Consultant in Herndon, VA. According to the post, the company is looking for someone “to work on the development of a big data framework platform for one of our Federal customers in Herndon. The selected candidate will apply his/her development experience against big data projects in combination with Agile practices in order to help the development team continuously innovate and incrementally deliver system functionality. Additionally, the candidate will be responsible for determining the appropriate data models, transformational technologies, and algorithms needed to advance the organization’s tradecraft.”

Qualifications for the position include the following: “20+ years of IT experience. Extensive experience in information systems development, functional and data requirements analysis, systems analysis and design, programming, program design and documentation preparation. Must have an active TS/SCI w/ full scope security clearance, or be eligible to receive one. US Citizenship is required. Experience or knowledge of Hadoop (map/reduce). Experience or knowledge of NoSQL technologies (such as HBASE, Cassandra). Experience or knowledge of NoSQL Data Warehouse technologies (such as HIVE). Experience using Open Source Search Engines and writing Queries against large data holdings (such as Lucene). Experience working with very large datasets on clustered environments .”

Learn more and apply here.

photo credit: Evans & Chambers

A recent article discusses a great resource for businesses and individuals interested in NoSQL technology: “The NoSQL movement had hit its stride. But for those of us on the outside, the conversations about Cassandra, HBase, Hive and MongoDB are hard to parse, maybe even unintelligible. But if you really want to know, there’s a website that can help. The NoSQL Tapes is a collection of videos shot by Tim Anglade in a journey around the world made last summer to discuss what was happening in data storage and access.”

It continues, “The tapes, which have attracted 17,000 unique visitors to the site, are well done and can offer a good overview of technologies from specific (Mike Miller on MapReduce) to general (Ben Black on NoSQL and cloud computing). They are helpful primers on their various topics, but they also capture a moment in tech history when a disparate group of engineers got together to build tools using open source software. The videos showcase the use cases for technology as well as attempt to engage people on the technical merits of each project, as opposed to a bunch of marketing speak.”

The article adds, “As Anglade told me in a conversation, these videos represent a time before NoSQL ‘moved toward the enterprise’ and became overrun with marketing. Now an evangelist at Cloudant, which offers a platform for CouchDB, Anglade recalls his summer of running around interviewing technologists fondly. The site launched in January, and so far Anglade has put up about 20 videos and has about 20 more to process and put on the site (although not all of those may make the grade). At about an hour-long, the videos on the site represent a huge time commitment, and the NoSQL community has been the primary audience.”

photo credit: The NoSQL Tapes

A recent article delves into the value of a strong implementation strategy for master data management efforts. The article begins, “The aim of master data management (MDM) is to create a single source of all enterprise information (customer profiles to support customer data integration, services codes, trading partners, company hierarchies, etc). It guarantees consistent and accurate data, as well as assist in the business processes. However, the success of the application also depends on how effective the implementation plan is. If the steps are well defined and they are understood among everyone involved, it would be easier for the application to be integrated into any company. This is why it is very important to formulate and come up with a clear master data management strategy.”

It continues, “Primarily, the development of policies that concerns MDM begins with the identification of factors that negatively affect the business workflow.Companies who spend time building team members to be effective and In this way, the IT people will know what to fix or how to design a plan that is deemed to work competently and a plan that will pave the way to increased productivity, employee efficiency, and a higher level of customer satisfaction.”

The article adds that MDM software “is more than just collecting and organizing data assets, this should also be implemented to address the current and future needs of the business. It must be used to increase profits, build long-term relationship with trading partners, reduce or totally eliminate problems about employee productivity and other related business goals. Having said that, a master data management strategy that is seen to work and heighten MDM capabilities is coming up with measures that will produce high quality of information. Having reliable data at all times will help bring a solid customer base and build company integrity.”

photo credit: Lee Cannon

Joab Jackson reports that “Users of NoSQL databases and data processing frameworks such as CouchDB and Hadoop are deploying these new technologies for their speed, scalability and flexibility, judging from a number of sessions at the NoSQL Now conference being held this week in San Jose, California.” He explains, “EMC is using a mixture of traditional databases and newfangled NoSQL data stores to analyze public perception of the company and its products, explained Subramanian Kartik, distinguished EMC engineer, during one talk. The process, called sentiment analysis, involves scanning hundreds of technology blogs, finding mentions of EMC and its products, and assessing if the references are positive or negative, using words in the text.”

Jackson adds, “For digital media company AOL, NoSQL products provide speed and volume that would not be possible using traditional relational databases. The company uses Hadoop and the CouchDB NoSQL database to run its ad targeting operations, said Matt Ingenthron, manager of community relations for Couchbase, during another talk. AOL has developed a system that can pick out a set of targeted ads for each time a user opens an AOL page. What ads are chosen can be based on the data that AOL has on the user, along with algorithmic guesses about what ads would be most of interest to that user. The process must be executed within about 40 milliseconds.”

Read more here.

photo credit: EMC

LucasArts is looking for a Senior Backend Engineer (Platform) in San Francisco, CA. Responsibilities of the position include the following: “Take ownership of major features and components of our back-end system. Work closely with game designers, game programmers and QA to deliver industry-leading enterprise gaming solutions. Architect, design and implement major features and components from user stories. Drive major design decisions of software architecture and functionality. Stay up-to-date with new technologies and architectures. Help drive effective process, design, and coding standards for software development. Perform code and design reviews.”

Qualifications for the position include, “Experience with n-tier and service-oriented architectures. Expert-level knowledge in Java version 5,6 and Java concurrency OR equivalent experience in other languages. Experience with multiple programming languages a plus. Understanding or experience with NoSQL database solutions (Hadoop, Cassandra, etc.). Knowledge of relational database concepts and practices. Experience working with enterprise technologies, including web services, message-oriented middleware, security, and web development. Familiarity with various forms of testing practices, including regression, acceptance, functional, and load testing.”

Learn more and apply here.

photo credit: LucasArts

HP is searching for a Manager of Business Intelligence and Technical Support in Nashville, TN. According to the post, “The successful candidate must provide strong leadership to the BI and ETS teams and be able to exert indirect leadership and influence with other technical management staff. This position will also serve as a backup for the Technical Services Director. The Business Intelligence team is comprised of data warehouse, ETL, and reporting specialists responsible for maintaining, operating and enhancing data warehouse repositories, reporting tools and operational dashboards that provide our customer with robust business intelligence.”

It continues, “The technologies used and support by this team include BusinessObjects Xi suite, BusinessObjects Xcelsius, SQL, and C programming language. The Enterprise Technical Support team is responsible for providing technical direction for the Technical Services Group. The team is comprised of Application Architects, Configuration Management, and developers and data analysts. This position is also responsible for directing and managing analysis, programming and operational maintenance tasks for specific applications or systems. This group will be driving our client’s strategies and deliverables. The supervisory responsibilities include hiring/termination and performance and disciplinary management.”

Learn more and apply here.

photo credit: HP