Why do you need logs?

Logs are needed for different reasons. Your company may be a shop of developers, so you need a repository for your code, who checks out and who checks in part of the code, which ever version control system you use it needs to keep track of the different versions of code the developers produce. Look at it as a log system within the application. A competent system like that most definitely has logging capabilities (if not, throw it out now), so that is easy. Who did what at which point of time? 

That application needs to log who checked out which portions of code at which point of time, since if something out of the ordinary would happen, like a buffer overflow in the Linux kernel, it would be easy to see exactly when the faulty code was introduced and by whom. The "who" part is (hopefully) not there for blame reasons, but for educational as in "we all learn from errors". Still, the main reason for logging, and preserving the logs, is traceability.

The example above can be seen as a "Use Case". Use cases are a good way to understand where and what your needs are. What do you need to record, and on which grounds? Use cases can consist on almost anything, but the most important issue here is that the case will give your business something of value. Something that can be applied onto your particular situation that needs to be followed up on a regular basis. This could be something in way of (PCI-DSS) "where does my e-commerce data reside and flow?" (i.e. credit card numbers), or "who created an admin user in database x?", or "how come a user logged in from three different sites within five minutes?". 

For some reason you need logs from parts of your environment and you need to work out a few good use cases to have as a starting point. Once that is done you have a picture of what you want to see in a report. The next step is to look at your logs. Is there enough information in your logs to achieve what you need? If there is - fine. If there isn't - uh-oh… Don't despair, there is usually a solution. 

Let's look at syslog, built in to all Unix-like systems, routers, switches, firewalls and a lot of other devices. It comes with different levels of logging (0-7) from/to different facilities, and can be logged locally or remotely. Usually level INFO (6) can be considered as a sufficient level of logging (only debug is creating more output, sometimes a ridiculous amount of output) for investigations. There are several products for the Windows platform that converts event logs to syslog and sends them on to a central (or several) syslog repository where the logs can be saved and on demand be parsed at will.

Most applications have logging built-in in some portion. It is up to the vendor to decide what is logged, to which level, and whether this level can be configured or not. That is a shame. First and foremost it is up to the user to decide what needs to be logged, not the vendor. But as this is not the case you need to either accept it, file a bug report or take radical decisions as in throwing out the product or wrap it up in some kind of sandbox, which can take care of logging the input and output. The problem with "accept it" is that the product in scope probably will not meet the requirements that you, your organization nor the auditor have, and one of those may have a different opinion on the matter.

If you need logging to the extreme Big Brother experience where you log everything from an employee using the card to get into the premises, passing the cafeteria area, pouring a cup of double espresso, entering the work area, logging into his account, opening [x,y,z] applications, surfing to [a,b,c…] doing this and that until lunch, locking the computer screen, going for lunch… you see where this is going. You will probably end up in throwing out a lot of your existing software to comply with your needs. It's doable, but does it justify the cost?

This leads you to level 2, Collection; you are collecting logs, but you're really not looking at the logs. Ever. But you're collecting logs, which is a good thing in itself. So, Good On You!