#8 Create A Data Source in Studio - Type (Load By Script)
hello hi everyone so in this video we are going to create our data source in studio and then put our script there and then see uh test it see see what kind of result that we can get the first thing that we have to make sure is uh in here in this sheet in the script sheet you probably saw in my previous video we have this table and i have all this uh the length for all these coins okay and you um if you notice carefully i have different numbers now okay i don't have the same length for all coins uh that's that's not a mistake that's uh basically the api that we are using on on daily basis or probably in few days the number of the length is changing automatically so it is not changing drastically so we don't have like changes thousand five hundred to three thousand something like this uh we have some changes but it's more like we are adding few elements into the array okay it's three array so make sure that you have the right numbers here okay that's highly important so it's going to give you at this in the third column level the length that we should use in our script so these are the right numbers that we should have now in our script so let me show you the script that i have in my parse response body and you can see here i have updated all the numbers i have 120 120 83 and then 172 okay so so make sure that this is done uh at this point this is highly important when it comes to uh when you are going to load the data into your target table now we are creating a data source it is actually going to just load the data into our staging table but is we are not loading directly into our target table but now if you make sure that these numbers are correct uh you will be sure in the end that you have a good uh source code okay for your data source that is not there are no we don't have any errors basically if you see here i have the total uh 11843 so if you click on load data when you create um when you have done with the data source um you will have the message like success and then processed 11 843 you should have that number if you don't have that number you have some problem in your code okay that's that's why i'm showing you uh um first uh i'm giving you all this information first before uh before we start to create our data source okay so let's go now into studio click on create application file for data source click on create and give it a name i'm going to have coin history [Music] and detail let's say details data source okay so the next thing is going to be exactly the same name but i'm going to just delete data source and then put staging table because this is a label name for my staging table so i'm going to put here stating table okay so this is my um the database name for my staging table the type is going to be custom load by script and here we are going to add our script so let's go back here into our sheet data source load by script this is the one that we want i'm going here okay just going to ctrl a c let's go back here just paste our code here it's kind of messy i'm going to click at this level it's going to arrange everything okay so i have my code i have my coins i have my for loop and one thing i forgot to mention as well sorry uh we have created in our data source this um i mean we are getting the data for website url and then we have ids log symbol name your id and website url okay all this information but in my in flow designer if i go back let me find flow designer here in the first step for the first request uh i don't have uh initially i didn't have website url when we were creating we didn't have some of these okay make sure that you have here that you have id slug symbol name uuid and website url okay so you have to add few few uh let's few elements here and then make sure that you have also id symbol you id names like the website url here as well for your coin object and also in the output variable here in the end make sure you have all of them here as well okay that's a that's the second thing and the third one okay let's say this is uh status the status should be published otherwise float when you call the pro design action from your data source it is going to fail so make sure that it's published here okay so we have our code now we don't need to do anything else then save it okay and if you go to related links you can test load 24 cards to see if it is going to be successful but i'm sure this one is really good so i'm going to click on directly load or record okay so what is going to happen at this moment if everything is good you can see message creating table okay we are creating at this specific moment the table the staging table and it is inserting is it is creating as well all the fields that we have in our data source like you uh you underscore id these are the you underscore slug okay these are the database name for our fields so it is actually creating now and then also once it is done it is actually going to get the data and then put the data into our fields creating records okay so let's see it is going to take around two minutes right so we have a completion call here which is a success state is complete and we have the number here processed is eleven thousand eight hundred and forty three okay so if i go back i can show you the number here you go to script you can see here 11 843 so if you make sure in the beginning that you have the right numbers here it's all good we have all the data now let's see if we have uh the data is correct it's proper so let's click on import set this is our import set number okay the state is loaded and click on that and you have the tab here import set rows so click on that and then you have all the records here okay you can pick any one of them so i'm going to click here and see if you have something and great bitcoin you can see but website url it is undefined so i have some sort of problem here you website you underscored url so that's the one that i have here okay let me just check quickly what's going on here [Music] okay kind object so let me go back to flow designer and so i have website url probably that's a problem here that's the problem here the name of the okay so this should be website url with capital because because this is the name that we have in our history array this is how it is written so that's not a problem so the problem is the syntax that we are using here website urls that's not a problem that's just something that we have to rectify uh so let's go here and instead of having capital letter here i'm gonna have small letter okay finally it was not all good there was something to change here okay so now it should be proper so i'm gonna reload one more time okay i make sure that's good so click on load or record and then let's see if this time we are going to get the website url or not so you can see here for the second time it just we have the state running okay we don't have any other information here it's not creating cable or putting any uh creating any fields because these are it's already done great so we have the completion code here success and process is 11 843 that's the number that we want so let's go here into the input set so this time we have six uh you can see here the ending number here is 613. so this is the last one so i'm going to click on that and see if you have uh if you have our website url or not great so i have my website url so the problem was the naming that i had uh in my uh in my code okay so i have all the data that i want these are the historical data for the past five years i have it okay so the next thing what we are going to do is we are going to create our tables the custom tables coins tables and then transformer put the data and then we are going to create also uh crypto table business rule transfer map and then put the data okay we have only one data source for both table for now see you in the next video thank you for watching
https://www.youtube.com/watch?v=ZJ0W2jF52bU