What is DTLoggedExec?
DTLoggedExec is a tool that allows you to run a Sql Server Integration Services (SSIS) Package producing a full and detailed logging informations of execution status and package runtime data, including dataflow profiling information. In brief it allows to fully
log and instrument package execution. Here's some samples of what you can do with all the data that DTLoggedExec is able to produce at runtime for any package, new or existing:
Why you created DTLoggedExec?
If you're using Sql Server 2005/2008 Integration Services, you should know that the packages you've created can be launched via DTExec or SQL Server Agent. In both cases you'll find that, though an extensive logging infrastucture seems to be present,
logged information doesn't really help you to understand why and how a package that has always run correctly someday decides to go wrong.
In addition to this big limitation you may also have already discovered that logging can't just be switch on or off right from the execution tool. Though it seems possibile, if you want to log on a text file your package MUST contain a connection manager
that points to the file you want, otherwise logging cannot be used. This means that you should engineer your packages so that they support the logging infrastructure, BEFORE you need it. This also means that you have to take care of preparing the way for logging
in ALL of your packages.
This is really not how logging should work for me. I need something that can be switched on or off as I desire, without having me to do anything during package development; in addition I need ALL the data and the information I can have, since when I have to
understand why a package didn't run succesfully in a production enviroment, well, I can't run the package in debug mode from Visual Studio right from the production server (otherwise I'll be doing debugging on production machines, not an healthy
way of working from my point of view)!
Another missing feature that I really missed a lot is the ability to profile dataflow tasks. To do a proper profiling you should add "Row Count" wherever you need to know how many rows has been processed, take the start and the end time of each transformation
and put everything in a database for later analysis. This is a highly time consuming way to work IMHO, since package instrumentation should be something provided right from the execution tool, and not by who creates the pakages. Also, putting the instrumentation
logic within the package make it more complex and long to develop.
So I decided to leverage all the power hidden in the Integration Services Object Model (which is really cool) and I created the DTLoggedExec tool, which aims to be a DTExec replacement, putting into it all the thing I need to work at my best.
What are the features of DTLoggedExec?
DTLoggedExec has a lot of interesing features:
Under which license is released?
- Enable / Disable logging on the fly for any package, just right from the command line
- Log Variables value
- Log Values of Properties bound to Expressions
- In case of error, log all properties along with related connection properties of the erroneous task. This is IDEAL for post-mortem debugging
- Fully pluggable architecture: decide where to log informations (natively supported: No Log, Console, CSV). Add your own log providers to customize logging
- Choose which event you want to log
- DTExec parameters emulation (not yet 100% finished)
- Compiled to support 32bit and 64bit platforms (32bit and 64bit executables are available in the zipped file)
- Support for Integration Services 2008 (Support of 2005 has been dropped from version 1.1.2008.4)
- DataFlow Profiling: log DataFlow performance data, how many rows each component has processed in how much time.
- Log to CSV
- Import Log and Profiling Data into SQL Server for further analysis automatically
- Add parameters support to packages
- Samples for all DTLoggedExec features (12 samples provided)
- Logging of events that happened during package loading
- Autoconfigurable packages
The code and the application are licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 Italy License:
The "NoDerives" licensing limitation just means that if you want to contribute to the project you have to join codeplex and/or contact me so that I can add you to official authors list.
The "NonCommercial" licensing limitation just means that you cannot take the code and the executable and sell it. Of course, if you want to use it in your production eviroment, that's fine.
Where I can see some screenshot and have more some help and/or more information