The Hidden Costs of Big Data

Name: Pornthiya Tungkijrat (唐夕雅) 
Student ID: M0869827 

What are the hidden costs of big data, you ask? It is the costs that you could easily be overlooking within your big data management processes.

Big Data’s Obvious Costs
On the surface, the costs associated with managing big data may seem simple enough to measure.
You have obvious costs, such as:
  • The price of the software tools that you use to manage and analyze data.
  • The cost of storage infrastructure for your data.
  • The cost in terms of staff time that your company’s data engineers spend managing data.
Those costs are easy to identify and calculate.

Big Data’s Hidden Costs
If you stop your big data cost calculations with the factors listed above, however, you won’t achieve a comprehensive analysis of the true cost of your data management operations.

You’ll be missing a variety of hidden costs, such as:

Inefficient data integration
Deriving value from your data typically requires you to transform and integrate it. If your data integration solution involves a lot of manual effort, redundancy or other types of inefficiency, it could be a major source of hidden costs in the form of wasted staff time and unnecessary infrastructure. (Syncsort’s Big Data solutions can help address this hidden cost by streamlining data transformation and related processes.)

Networking data costs
Transferring data over the network can feel free because in most cases, the cost of bandwidth is so low as to be negligible. But when you are dealing with terabytes of data, those costs can add up. Whether you are only transferring data locally on your internal network, or you are moving data between a cloud and on-premise infrastructure, the networking devices and bandwidth that you’ll need have a price.

Unnecessary backups
You should always back up your data. However, if you are backing up your data more frequently than you require or storing more copies than you need, you are not operating in the most cost-effective way possible. You want to ensure that you have the right level of data backups, but not more.

Low-quality data
Data that is rife with data quality problems will likely cost you a lot more to store, integrate and analyze than data that is free of errors. While avoiding data quality problems entirely may not be possible, correcting data quality issues via automated tools can help you to minimize the costs you incur due to low data quality.

Uninformed employees
Are your employees citizen data scientists? Or do they totally lack an understanding of data management best practices? In the latter case, the lack of knowledge on the part of your employees could bloat your big data costs by creating bottlenecks and data quality errors whenever someone who is not a professional data scientist touches data. Avoid this hidden big data cost by turning your employees into citizen data scientists.


Comments

Post a Comment

Popular posts from this blog

How Big Data Can Boost Weather Forecasting

張岳庭 D0741017 Source:  https://www.wired.com/insights/2013/02/how-big-data-can-boost-weather-forecasting/ Considering the number of variables involved and the complex interactions between these variables, weather forecasting has always been extremely challenging. The significant increase in data collection and processing capabilities has greatly improved the ability of weather forecasters to pinpoint the time and severity of hurricanes, floods, snowstorms and other weather events. Globally, more and more evidence of climate change is prompting governments and scientists to take action to protect people and property from its effects. But to take effective measures, they need to know more about the weather-from everything that will happen tomorrow to what will happen next year. According to the professor of economics and finance , Alan Anderson, PhD , “Example of an application of big data to weather forecasting is IBM’s Deep Thunder. Unlike many weather forecasting systems...
Read more

How Big Data is Changing the Production Industry

Blog#4 : How Big Data is Changing the Production Industry Name : Aileen 蔣慧玲 Student ID : D0726917 Source :   https://www.productionhub.com/blog/post/how-big-data-is-changing-the-production-industry Information technology plays a major role in Hollywood. Producers have to use big data analytics to get the right consumers and communicate with them well. Big data technologies will help filmmakers figure out the best ways to share their ideas and deliver messages, develop movies and content that involve ethical and societal issues. Therefore, information that we can get from the help of big data can help filmmakers make key decisions. Big Data is Changing the Production Game Big data technology can help filmmakers to connect and engage with consumers more directly using social media, mobile devices and traditional advertising outlets. Using this technology, producers can develop promotions that reaches the right consumer at the right time. It is not enough just to mainta...
Read more

Big Data case study: 5 relevant examples from the airline industry

D0740712 Connie  吳霈晴 Website: https://blog.datumize.com/5-relevant-examples-of-a-big-data-case-study-from-the-airline-industry With the summer vacation coming, travel is the one that people love to do during vacation, especially long vacation, going abroad may be the first choice for many travelers. So this makes me think of what is the relationship between big data and airplanes? How will it apply on the airline industry? This article tells about the benefits of big data on airlines, which covers three main advantages. 1.      Smarter maintenance 2.      Safer flights 3.      Improve service The first one includes the fuel of airplane, by analyzing and predicting, the article takes some specific airplane company as an example, giving the real number and how much they save the cost. Second, safety. Like this kind of transportation has a higher risk to take, if have the big data’s help, th...
Read more