Choosing Mr. Right

As you can imagine from the above, whatever products you choose, do not trust a vendor that promises "Plug it in and you will be up and running in ten minutes!". This will not happen. You will need to tweak and configure until your fingers bleed, your eyes are blood-shot from crying, and your vocabulary is out of new and innovative cursing as well as the old and proven four-letter words before you feel things are under control. Sometimes that feelings is good, as in: you're on the right track. The rest is just a false prediction of what is ahead of your adventure with log collection and reporting.

Now, don't let these words put you off. Once you've overcome the first hurdles and caveats you might experience the beauty and necessity of logs. Sounds crazy, but when you understand the heartbeats and almost organic life that is going on in your data centers you will see the benefits of your correlated and filtered data logs, and the advantage it gives your organization in reacting almost proactively to those alerts and reports you are able to squeeze out of those hundreds of millions events that might occur during a normal day at an average site with a few active databases.

Your mission is to choose the right product(s) for your needs. So you need to pin out your needs first. In cases of regulatory requirements it's pretty straight forward. Take PCI-DSS (I'll use that as an example since I am very familiar with those requirements after working for years with them), the requirements (PCI-DSS 2.0) clearly states that you need to collect logs and see to it that they can't be tampered with. There are also requirements of how often logs are to be reviewed. So you need to decide to what extent you want to obey, or follow, these requirements. As you will see later, maybe it is a good idea to look beyond the requirements and not only to comply, but also look at what good will the requirements do for you and what advantage you can gain from using them as an instrument in your daily work environment.

Let's start with your requirements. Don't worry about other's at this stage. This is your mission and your requirements comes first, and only later we'll take other requirements into consideration. It might just be that they walk hand in hand, or needs just a little tweak or persuasion so they will tango. 

The main mantra in logging is:

Who did what at which point of time? Who did what at which point of time? Who did what at which point of time? 

Is that clear enough? In one word: Traceability.