Consuming Data

Nov 3, 2010 at 2:30 PM


Great tool, really very useful. I've read the information on the page 'Consuming Profiled Data' but I was hoping for some further detail on loading the data through an SSIS package rather than dropping the file on to the .bat script.

How are people loading these file automatically? (*.dtsCSVLog files in a directory) - I've looked at for each loop in SSIS but I can't get it to pass the filename as an argument to the bat file (using execute process task).



Nov 4, 2010 at 12:44 PM


I've created a .bat script that uses the code in import_all_logs.txt and import_all_profiles.txt but changed the % to %%. I'm then using an Execute Process Task in SSIS to load the files.



Nov 4, 2010 at 1:21 PM

Hi JW5

I've decided not to load logged data into the database using SSIS becacuse otherwise how you can also monitor that those packages run correctly? This is a classic situation that would lead to an infinte loop :) hence the idea to use a very simple BCP to load everything into SQL Server for further analysis.

I usually use Powershell to automate the loading of logged data, but of course the option to orchestrate the processes using SSIS and only the Execute Process Task is a another good option.