#Part 16 Remove Duplicates Records
okay so in this video we are going to talk about data cleaning why did actually name what I mean by data cleaning here so so far what we have done we have all our data in our table we have our historical data and we have our and I mean we have our daily summary the next part will be working on towards I mean working with performance and analytics and creating in the dashboard before doing anything what we need to do we have to make sure that we have an accurate data okay so what do you see here this is a score sheet once you have done with the performance analytics you can see this Koshi I'm showing you this in advance just to give you some details you can see here I have another Kingdom okay United States of America USA Venezuela and village Venezuela Bolivarian Republic okay so we have duplicates and we need to replace I mean we should have only one country okay for United States for example so the United States of America is created while we are loading the data of the historical data and USA was created when we were loading the daily summary so what we need to do at this point is just go through our table and look for United States of America all the records and replace it by USA and we have to do for all the countries where we have the duplicates so I put in the description or in the resource section depending on the platform posting this video we have a file where you can actually see all the duplicates there are around 15 countries not more than that so it takes around 20 minutes to do the data cleaning so all I need to do go through all these countries and replace it okay United States of America by USA United Kingdom by UK and so on okay so I put the countries in the description sorry I in the file so you can use that one the second thing we don't have data for China and Australia unfortunately the API is not giving us any data so I cannot do much more but what I can do I have another instance where somehow I took the data for China and Australia I guess so it's going to be an optional section probably I will have after in the end once we have done with the course probably there will be another section I will be updating there a tour there will be an excel file sure that you can have it now to load this historical data for China or I will create a scripted API that you can use in order to send a request to the to my to another instance that I have where you can send the request to pull the data and then put the data into a table that's um this is completely optional I'm not sure that will be done in a certain time so yeah the third thing is we do have some real cars where we don't have the location here okay if you have a lot around to record so all you need to do is to run a query in your table and then see where you don't have any location so you just go to location field type equal norm and press enter it's going to give you to to record and you can delete them okay that's it now let's see how we can for example replace for where we have a duplicate okay so this is a table where I have other calls for example here for Iran Islamic Republic this is a historical data this country was created then I when we imported the historical data okay and the current data is Iran so I should replace all of this by Iran so if you try to select everything and then replace it is going to give you an error security province right with its people because when we were creating our design form I mean when we were editing or creating the fields in our design form form we said the location field to read-only so we have to uncheck that box that's what we are going to do right now okay this is our design form and you can see here there is the location film click on the and the Settings icon uncheck the read-only box and say let's say now select all of the records here I mean all the the column location column select everything and then right-click that's all right just click on it and then change it to Iran and then select the country that's it so this is a way we can actually replace for example Iran Islamic Republic of Iran so you have to do the same thing for all countries there are not there are only in 15 countries so that's fine so thanks for watching so this was our session for how to clean the data in our table
https://www.youtube.com/watch?v=A5iND0SwhQM