logo

NJP

#Part 13 Load Daily Summary Data Into The Staging Table

Import · Jun 09, 2020 · video

hello everyone so in this video what we are going to do is to brighter data's create a data source and this is going to be load bi script type data source the goal here is to call the flow design our action that we have created before and once we have the outputs so as we have outputs in our flow designer action we are going to take these outputs and then use those variables in order to send those the values that we have in our outputs in into our staging table so while we are creating the data source we are going to also have our staging table so let's see that in action now so let's create our data source so let's look for data source here click on create so the first thing here I'm gonna give a name so daily summary data import and I'm going to have also my import set table which is my staging table here so this one and the type is going to be custom load bi script okay we're going to write a script and click on summit so we are going to just create the data source now now let's write the script so the first thing is just take the code snippet that you have in your flow designer action ok the following codes then just paste it here this is the first part calling the flow designer action and then we will have the result in each different outputs that we have here son and this is going to be summary one array summary two array and take input array okay the next thing as we know these are arrays we are going to run a loop okay on this arrays and it will the loop will be ending it will go until the it will take the entire length of the errors here okay so the bar I equal to zero are inferior smaller than somebody won all right so we can take any area that we want here okay and then what we are going to do is basically accessing the values that we want we won't come from cases we want to cover cases we want dead cases we want date and country so I'm going to create here for example wall total country so this is the variable local variable and then I have the array and I'm accessing the values using the properties if you remember we have summary won't array and then an object and then we have properties like contain cases dead cases and we have somebody to array around and then we have object like dead cases and country and we have the same thing for example for date so we are basically creating local variables like total confirm total recover total deaths and we are accessing the variables by using the following syntax here summary to array for the total deaths and then we are going to access in death okay this is the property that I have created sorry it was dead cases this is a property that I have created before and I'm going to say doing I'm doing the same thing for country I'm going to do the same thing or the date so basically taking all the values that we have in our outputs but with the puppet with this syntax basically the object and then the properties that we have in these objects and taking those values and storing it in each individual local variables like total confirmed totally covered but all of that in our country okay and also for date so and after that what we are going to do within the same loop is we are going to basically create fields in our staging table and having those values okay we gonna send those values to those to these specific fields so total confirm total capital that country and date we will have to send that to those fields so we are creating here an object called map which will have the followings the following property so it's going to be you underscore Toller underscore confirm it will have for example total confirmed cases and then I'm going to apply the same logic for example for total recovered for total debt but all a country and date okay so this is what we are creating an object with these pairs okay property and values okay so name values pairs okay so this is like the map that we are creating this is a map object and the the Etowah underscore you total underscore confirm you total underscore recovered these are the field names that will that I will have for my staging tables that's why I'm naming it like this and then we are going to use basically a method called insert which will basically insert the map the map object into our staging table so I'm going to write import set table dot insert and we mentioned in 2010 sees the map object and what we it will do is basically creating rows for my staging tables okay for my staging table so it's going to create the first row with the values of total confirm to recover that country and date with the fields name you underscore total front we know and so on and the first second row it will add values so each time it will add values it basically creating rows into my staging table so this is the entire code that we have to write and if you want to insert take the values from external source for example here from from output given by flow designer action so now we have done with our script and it's time to test if this is going to actually insert all the recalls into our staging table or not so what you can do for that basically if you scroll down that's one and you will have load all records okay click on load all records you can do a test if you want and then this is will start to create a import set and creating the staging table and you can see we have a success message here we have successfully loaded some data into our staging table ok we have created the table and incident table now let's click on import set and see in the import set rows so we have all these rows created we have record rate okay this is the basically the result that we should get ok we have the result for example here for Australia and yeah total confirmed cases total cover cases total dead and we have the date okay the next thing is to know we are going to do create a transform app you know to take the data that we have in our staging table and then put it in our target table which is going to be our code in 19 cases table thank you for watching

View original source

https://www.youtube.com/watch?v=-9HbllfMUT0