Data Transformation for Machine Learning

Title : Data Transformation for Machine Learning

Blog #1

Name : Devina
Student ID : D0731576
Source : https://insidebigdata.com/2020/05/07/data-transformation-for-machine-learning/
Written By : Damian Chan, Technical Success Manager, Matillion


Nowadays, everyone is talking about machine learning including industry experts, competitors, and customers. Human can use machine learning that usually understood to be a form of artificial intelligence to build and train model to process data, which can help us to have a better prediction and also allows computer systems to learn from data to make decisions without being explicitly programmed to do so.

Machine learning can learn and improve their analysis without reprogramming based on the given data. Deep learning is also a subset of machine learning that involving artificial neural networks.

In order to get a good insight you have to provide a good data to support machine learning to analyze your model. Data can be really messy or even they called it as a daunting tasks. Data transforming can be time consuming and tedious without the right technology stack in place.

How data transformation can improve machine leaning, here are some key aspects :
1. Remove Unused and Repeated Columns not only improve speed on model trains but also when        you analyze it
2. Change Data Types to save memory usage
3. Handle Missing Data by considering imputation to replacing the missing value with a simple         placeholder or another value, based on some kind of assumption
4. Remove String Formatting and Non-Alphanumeric Characters although removing formatting and other characters makes the sentence less readable for humans, this approach helps the algorithm to better digest the data
5. Convert Categorical Data to Numerical converting values such as yes and no to 1 and 0
6. Convert Timestamps define specific data/time format and convert all timestamps to the defined format

Machine Leaning can help your business process and understand data insight faster. But in order to transforming data for analysis it can be hard based on the growing volume, variety and velocity of big data, so to overcome this you need ETL software that is purpose-built for the cloud.

Comments

Post a Comment

Popular posts from this blog

How Big Data Can Boost Weather Forecasting

張岳庭 D0741017 Source:  https://www.wired.com/insights/2013/02/how-big-data-can-boost-weather-forecasting/ Considering the number of variables involved and the complex interactions between these variables, weather forecasting has always been extremely challenging. The significant increase in data collection and processing capabilities has greatly improved the ability of weather forecasters to pinpoint the time and severity of hurricanes, floods, snowstorms and other weather events. Globally, more and more evidence of climate change is prompting governments and scientists to take action to protect people and property from its effects. But to take effective measures, they need to know more about the weather-from everything that will happen tomorrow to what will happen next year. According to the professor of economics and finance , Alan Anderson, PhD , “Example of an application of big data to weather forecasting is IBM’s Deep Thunder. Unlike many weather forecasting systems...
Read more

How Big Data is Changing the Production Industry

Blog#4 : How Big Data is Changing the Production Industry Name : Aileen 蔣慧玲 Student ID : D0726917 Source :   https://www.productionhub.com/blog/post/how-big-data-is-changing-the-production-industry Information technology plays a major role in Hollywood. Producers have to use big data analytics to get the right consumers and communicate with them well. Big data technologies will help filmmakers figure out the best ways to share their ideas and deliver messages, develop movies and content that involve ethical and societal issues. Therefore, information that we can get from the help of big data can help filmmakers make key decisions. Big Data is Changing the Production Game Big data technology can help filmmakers to connect and engage with consumers more directly using social media, mobile devices and traditional advertising outlets. Using this technology, producers can develop promotions that reaches the right consumer at the right time. It is not enough just to mainta...
Read more

Big Data case study: 5 relevant examples from the airline industry

D0740712 Connie  吳霈晴 Website: https://blog.datumize.com/5-relevant-examples-of-a-big-data-case-study-from-the-airline-industry With the summer vacation coming, travel is the one that people love to do during vacation, especially long vacation, going abroad may be the first choice for many travelers. So this makes me think of what is the relationship between big data and airplanes? How will it apply on the airline industry? This article tells about the benefits of big data on airlines, which covers three main advantages. 1.      Smarter maintenance 2.      Safer flights 3.      Improve service The first one includes the fuel of airplane, by analyzing and predicting, the article takes some specific airplane company as an example, giving the real number and how much they save the cost. Second, safety. Like this kind of transportation has a higher risk to take, if have the big data’s help, th...
Read more