Firewall Log Review and Analysis
After the decision has been made to log events from your firewall, the next step is
determining what you should be looking for in the logs and how you should properly
perform log analysis. The most important thing to remember is that firewall logs are
virtually worthless if no one ever looks at the logs. Logging is merely a means to an end,
namely knowing what is going on with your firewalls so that you can respond
accordingly. Review of the logs should not be reserved for only when an incident has
occurred. It should be a part of the weekly, if not daily, tasks that the firewall
administrators perform. To help reduce the time and effort required to review the logs,
many of the enterprise security incident management products provide tools and utilities
that assist the firewall administrator in separating the wheat from the chaff, allowing the
firewall administrator to spend less time reviewing the logs, while still providing the
information necessary to help identify situations before they become a problem.
Another aspect of reviewing the logs that should not be overlooked is the need to define a
log archive and normalization policy. Too many organizations do not store their firewall
logs long enough to adhere to regulations (some of which such as Sarbanes-Oxley are
generally accepted to require seven years of log data to be stored). This creates situations
where data from the logs may be necessary, but the logs themselves have been destroyed.
In conjunction with this, it is important to normalize your log data. Normalization just
means converting your logs into a standard format that allows for easier review and
correlation of data from different data sources (such as different firewall vendors).
What to Look for in Firewall Logs
After you have collected the firewall logs and begun the process of analyzing the logs,
determine the data that you should be looking for in the logs. With that said, it is
important to remember not to fall into the trap of looking in your firewall logs only for
"bad" events. Yes, firewall logs can be the key element in discovering security incidents
and compromises, but that is only one of the reasons for analyzing your logs. You also
want to be able to use the log information to assist in defining the baselines and normal
operations of the firewall. After all, one of the easiest ways to know whether behavior
that has been logged is malicious is to know what the good things are and then note the
exceptions.
The simple fact of the matter is that certain events should always raise suspicion when
they are detected. Ten of the most common events that warrant further investigation are
as follows:
• Authentication allowed.
• Traffic dropped (not addressed to the firewall).
• Firewall stop/start/restart.
• Firewall configuration changed.
• Interface up/down status changed.
• Administrator access granted.
• Connection was torn down.
• Authentication failed.
• Traffic dropped (addressed to the firewall).
• Administrator session ended.
The following sections explain these events in more detail.
Authentication Allowed
Although it may seem rather innocuous at first glance, it is important to look for
authentication-allowed events because they can identify situations where access was
granted by the firewall when it should not have been allowed. The reasons can range
from legitimate administrators logging on when they should not have to malicious users
logging on after compromising the account and password that they are using.
In addition, if your firewall is configured to authenticate user access, this event can be
used to identify users who have been authenticated for whatever function they are
attempting to perform.
Traffic Dropped (Not Addressed to the Firewall)
Most firewalls will have some resources that they are protecting. Traffic addressed to
these servers will typically be processed by the firewall and filtered accordingly.
Although traffic-dropped messages can indicate that someone is attempting to access a
protected resource in a manner other than what the firewall administrator has defined, a
common cause of this event is a simple misconfiguration of the ruleset. Therefore, if
users cannot access protected resources, it is important to review the logs to determine
whether the firewall is dropping the traffic, thereby pointing you in the direction of what
may need to be fixed to provide access to the resources requested.
Firewall Stop/Start/Restart
The firewall should never stop, start, or restart without the firewall administrator knowing
in advance that the situation is going to occur. This event can be caused by non-firewall-
specific issues such as power failures as well as by firewall-specific issues such as the
firewall crashing or a high-availability failover, and therefore it should always be
investigated in more detail to ascertain the root cause.
Firewall Configuration Changed
Almost all firewall configuration changes should be accompanied with the appropriate
change control documentation. This event always warrants further investigation to ensure
that the changes that were made are legitimate and in accordance with expected results.
In fact, many SIM products can be configured to perform a comparison of the changed
configuration against a "known good" configuration when a firewall configuration
changed event occurs. In fact, some products such as NetIQ Security Manager can
actually use that information to attempt to undo the changes that were made if they are
found to be out of compliance with the known good configuration.
Interface Up/Down Status Changed
Firewall interfaces transitioning from an up to a down status and vice versa can indicate
problems with the underlying network configuration. This information can prove
particularly helpful in situations where redundant firewalls are implemented, because the
network interfaces transitioning to a down state could cause the firewall failover process
to occur.
Administrator Access Granted
Whenever administrator access is granted, the corresponding event should be
investigated. Although similar to monitoring for authentication, in this case we are
looking explicitly at gaining administrator access. Most likely the access is expected, and
there is nothing suspicious or out of order that warrants further review. However, if that
is not the case, this event rapidly becomes an extremely high-priority situation that must
be investigated because the implication can be that an administrator account has been
compromised.
Connection Was Torn Down
The termination of connections is a relatively routine process that is a part of normal
communications. Where this event is particularly important, however, is in listing the
reason why the connection was torn down. For example, the connection may have been
torn down as a result of SYN timeout, which can be an indicator that someone is
attempting to cause a denial of service, especially if there are a lot of events of that
nature. In determining the cause of the connection tear down, it is important to review the
firewall documentation for the teardown causes. For example, Cisco Secure PIX Firewall
version 7.0 message ID 302014 lists the potential reasons for a TCP connection being
torn down as shown in Table 12-3.
Table 12-3. TCP Connection Teardown Reasons
Reason Description
Conn-timeout Connection ended because it was idle longer than the
configured idle timeout.
Deny Terminate Flow was terminated by application inspection.
Failover primary closed The standby unit in a failover pair deleted a connection
because of a message received from the active unit
FIN Timeout Force termination after 10 minutes awaiting the last ACK
or after half-closed timeout.
Flow closed by inspection Flow was terminated by inspection feature.
Flow terminated by IPS Flow was terminated by IPS.
Flow reset by IPS Flow was reset by IPS
Flow terminated by TCP
intercept
Flow was terminated by TCP Intercept.
Invalid SYN SYN packet not valid.
Idle Timeout Connection timed out because it was idle longer than
timeout value.
IPS fail-close Flow was terminated due to IPS card down.
SYN Control Back channel initiation from wrong side.
SYN Timeout Force termination after 2 minutes awaiting three-way
handshake completion.
TCP bad retransmission Connection terminated because of bad TCP retransmission.
TCP FINs Normal close-down sequence.
TCP Invalid SYN Invalid TCP SYN packet.
TCP Reset-I Reset was from the inside.
TCP Reset-O Reset was from the outside.
TCP segment partial
overlap
Detected a partially overlapping segment.
TCP unexpected window
size variation
Connection terminated due to variation in the TCP window
size.
Tunnel has been torn
down
Flow terminated because tunnel is down.
Unauth Deny Denied by URL filter.
Unknown Catchall error.
Xlate Clear Command-line removal
As you can see, reasons such as "Unauth Deny" or "Flow closed by inspection" can be
indicators of malicious traffic and warrant more concern and investigation than a reason
such as "TCP ResetI" (which is a normal method of applications terminating their
communications session).
Authentication Failed
Authentication-failed events can be indicators of everything from users making a typo
when they enter their password to malicious users making a brute-force attack in an
attempt to determine the password. Authentication-failed events should be examined in
particular detail when the account in question is a privileged or administrator-level
account.
Traffic Dropped (Addressed to the Firewall)
These events are similar to the traffic dropped that is not addressed to the firewall, with
the obvious difference being that in this case the traffic is addressed to the firewall. As a
general rule, the firewall should not have any traffic addressed directly to it on the
external interface; instead, all traffic should be destined for the resources being protected
by the firewall. These events can be indicators of malicious users attempting to gain
access to the firewall or a misconfiguration of things such as ICMP, IPsec, or
management or routing protocols and therefore should be investigated in more detail to
determine the exact nature of why the traffic was dropped.
Administrator Session Ended
Similar to administrator access being granted, administrator sessions ending should be
monitored to ensure that the administrator who had access was supposed to have access.
This type of event can also be used as a time benchmark because only administrators
should be able to make changes to the firewall, and therefore the logs should be
investigated in more detail for the time preceding the administrator session ending to see
exactly what commands may have been run.
Cisco Secure PIX Firewall Syslog Event Baseline
The following syslog events constitute a good baseline of events that should be
monitored and paid careful attention to in most environments. In essence, this list is here