Two of the biggest problems with your Machine Learning model can be either bad algorithm or bad data. But in what all forms can data be poor? Let’s take a look:
Insufficient quantity of data: Although it not so easy or cheap to gather large amounts of data, but generally, more the data fed to the algorithm, better will it learn and generalize. This is even more necessary with image or speech data and using Deep Learning on it.
Non representative training data: You may have a lot of data, but what if the data in real world is very different? Your model will pass all the training validations but will fail miserably on live data. Your training data needs to be a very good representative of real world data.
Biased data: You do a poll on LinkedIn to see if people think Online Education is better than Schools. Maximum people vote for online education. So can you conclude it’s better? Probably not. Your data is biased as you only asked LinkedIn users. Same goes for ML models. Data fed shouldn’t be biased to a specific region/type/category etc.
Poor quality data: Data full of errors, noise, missing values, outliers make it harder for the model to work. And fixing data does take a lot of time.
#datascience #machinelearning