It is hard to have a working centralized logging environment when you run network devices. Every vendor has his own version and understanding of syslog. Additional most did not speak any kind of structured log format. Some speak some binary format.
With that knowledge and the wish to include Cisco ASA and Nexus devices in your central Graylog, you will need some preparation. Most Important, this need to be done on all devices. But please adjust the settings that need to be changed. The goal is to have messages that have the most value and contain all information that is needed.
For Cisco devices that would look like the following message.
<189>91: *Mar 15 2018 21:48:41.663 UTC: %SYS-5-CONFIG_I: Configured from console by cisco on vty0 (192.168.200.1)
Prep on devices
The steps to get this configured on the Cisco cli, including NTP time sync and the Syslog target set could be similar the next code block. But be aware that you need to adjust the NTP server and the Syslog target to your local needs and requirements. If you change the timezone, you need to adjust settings in the processing in Graylog!
Cisco ASA
This config assumes that all packets are sourced from the management interface of the ASA.
clock timezone UTC 0 0
no clock summer-time
ntp server 0.0.0.0 prefer source management
logging timestamp
logging trap 6
logging enable
logging host management 1.2.3.4 tcp/8514
Cisco Nexus
NX-OS only supports Syslog via UDP! If you have Nexus devices, you need to adjust the processing rules and add a second input in the first rule.
clock timezone UTC 0 0
no clock summer-time
ntp server 1.3.4.5 prefer use-vrf management
logging timestamp milliseconds # even microseconds possible
logging server 1.2.3.4 port 8514 use-vrf management
logging source-interface mgmt 0
logging level all 6
logging origin-id hostname
Classic IOS
clock timezone UTC 0 0
no clock summer-time
ntp server 1.2.3.4 prefer
ntp update-calendar
service timestamps log datetime msec show-timezone localtime year
logging host 1.2.3.4 transport tcp port 8514
logging origin-id hostname
logging trap 6
Prep on Graylog
Create a RAW/Plaintext Input in Graylog and get the Input ID. This easiest way would be to select "show received messages" on the input page and copy the ID from the search bar. This will look like 5a71ae996c25ad4b80fbc085 as this is the UUID of the input.
Create in System > Grok Patterns the pattern that is needed in the processing. Click the green Create pattern and add the following.
NAME: CISCOTIMESTAMPTZ
PATTERN: %{CISCOTIMESTAMP}( %{TZ})?
NAME: NEXUSTIMESTAMP
PATTERN: %{YEAR} %{MONTH} %{MONTHDAY} %{TIME}( %{TZ})?
Create two Lookup Tables in Graylog. Over at System > Lookup Tables create a data adapter ciscos-severity-data and a data adapter cisco-ios-data - the files need to be located on all Graylog servers at the same location. For both need a unique cache - currently I would use a memory cache on a Graylog Node base. If they are named ciscos-severity-cache and cisco-ios-cache it is easy to identify them. Now just create the Lookup Tables, one that is called ciscos-serverity-lookuptable and the other cisco-ios-facility. If the names are changed, the pipeline rules need to be changed too.
The Content for the facility lookup can be found in this gist and here in the same gist. The setup of the Lookup Tables is described in the documentation of Graylog.
Create pipeline rules
We need a few rules that are chained in a pipeline. The first will just identify the Cisco messages - so this is the rule you need to adjust.
rule "cisco (1) flag cisco messages"
when
// -------------------------------
// Input MUST be a RAW/Plaintext Input
// With a Syslog Input this will not work!
// -------------------------------
// The following Configuration for logging of the Cisco Devices
// is what is expected. All other Configurations might not
// work
// -------------------------------
// # clock timezone UTC
// # no clock summer-time
// # ntp server 0.0.0.0 prefer
// # ntp server 0.pool.ntp.org
// # ntp server 1.pool.ntp.org
// # service timestamps log datetime msec show-timezone localtime
// # service timestamps debug datetime msec show-timezone localtime
// # logging source-interface Loopback0
// # logging host 0.0.0.0 transport tcp port 8514
// # logging trap 6
// ----------------------------------------
// messages will look like
// <189>91: *Oct 16 2010 21:48:41.663 UTC: %SYS-5-CONFIG_I: Configured from console by cisco on vty0 (192.168.200.1)
// ----------------------------------------
// one possible option is when you use a single
// input for your cisco devices
// other could be a specific IP range or any other
// identifier
// this is only done once - all other rules
// work with the created field from this rule
has_field("gl2_source_input") AND
to_string($message.gl2_source_input) == "5a71ae996c25ad4b80fbc085"
then
set_field("cisco_message", true);
end
rule "cisco (2) grok extract message"
when
has_field("cisco_message")
then
let message_field = to_string($message.message);
// the following GROK Patterns are needed in the System
// create them in 'System > Grok Patterns' if not present
// NAME: CISCOTIMESTAMPTZ
// PATTERN: %{CISCOTIMESTAMP}( %{TZ})?
// NAME: NEXUSTIMESTAMP
// PATTERN: %{YEAR} %{MONTH} %{MONTHDAY} %{TIME}( %{TZ})?
// one of the following pattern will work on IOS messages
let ios_1 = grok(pattern: "%{SYSLOG5424PRI}(%{NUMBER:log_sequence#})?:( %{NUMBER}:)? %{CISCOTIMESTAMPTZ:log_date}: %%{CISCO_REASON:facility}-%{INT:severity_level}-%{CISCO_REASON:facility_mnemonic}: %{GREEDYDATA:message}", value: message_field, only_named_captures: true);
let ios_2 = grok(pattern: "%{SYSLOG5424PRI}(%{NUMBER:log_sequence#})?:( %{NUMBER}:)? %{CISCOTIMESTAMPTZ:log_date}: %%{CISCO_REASON:facility}-%{CISCO_REASON:facility_sub}-%{INT:severity_level}-%{CISCO_REASON:facility_mnemonic}: %{GREEDYDATA:message}", value: message_field, only_named_captures: true);
// one of the following pattern will work on NEXUS messages
let nexus_1 = grok(pattern: "%{SYSLOG5424PRI}(%{NUMBER:log_sequence#})?: %{NEXUSTIMESTAMP:log_date}: %%{CISCO_REASON:facility}-%{INT:severity_level}-%{CISCO_REASON:facility_mnemonic}: %{GREEDYDATA:message}", value: message_field, only_named_captures: true);
let nexus_2 = grok(pattern: "%{SYSLOG5424PRI}(%{NUMBER:log_sequence#})?: %{NEXUSTIMESTAMP:log_date}: %%{CISCO_REASON:facility}-%{CISCO_REASON:facility_sub}-%{INT:severity_level}-%{CISCO_REASON:facility_mnemonic}: %{GREEDYDATA:message}", value: message_field, only_named_captures: true);
// write the fields that are extracted
set_fields(ios_1);
set_fields(ios_2);
set_fields(nexus_1);
set_fields(nexus_2);
end
rule "cisco (3.1) correct timestamp IOS"
// we want to create ISO8601 Timestamps
// make 'Feb 15 2015 13:33:22.111 UTC' ISO8601
when
has_field("cisco_message") AND
has_field("log_date") AND
grok(pattern: "%{MONTH} %{MONTHDAY} %{YEAR} %{TIME}", value:to_string($message.log_date)).matches == true
then
let time = parse_date(value:to_string($message.log_date), pattern:"MMM dd yyyy HH:mm:ss.SSS", timezone:"UTC");
set_field("timestamp",time);
end
rule "cisco (3.2) correct timestamp IOS (single number day)"
// we want to create ISO8601 Timestamps
// make 'Feb 5 2015 13:33:22.111' ISO8601
// cisco did not use 05 but <space>5 for days with a single digit
when
has_field("cisco_message") AND
has_field("log_date") AND
grok(pattern: "%{MONTH} %{MONTHDAY} %{YEAR} %{TIME}", value:to_string($message.log_date)).matches == true
then
let time = parse_date(value:to_string($message.log_date), pattern:"MMM d yyyy HH:mm:ss.SSS", timezone:"UTC");
set_field("timestamp",time);
end
rule "cisco (3.3) correct timestamp NEXUS"
// we want to create ISO8601 Timestamps
// make '2015 Feb 15 13:33:22.111' ISO8601
when
has_field("cisco_message") AND
has_field("log_date") AND
grok(pattern: "%{YEAR} %{MONTH} %{MONTHDAY} %{TIME}", value:to_string($message.log_date)).matches == true
then
let time = parse_date(value:to_string($message.log_date), pattern:"yyyy MMM dd HH:mm:ss.SSS", timezone:"UTC");
set_field("timestamp",time);
end
rule "cisco (3.4) correct timestamp NEXUS (single number day)"
// we want to create ISO8601 Timestamps
// make '2015 Feb 5 13:33:22.111' ISO8601
// cisco did not use 05 but <space>5 for days with a single digit
when
has_field("cisco_message") AND
has_field("log_date") AND
grok(pattern: "%{YEAR} %{MONTH} %{MONTHDAY} %{TIME}", value:to_string($message.log_date)).matches == true
then
let time = parse_date(value:to_string($message.log_date), pattern:"yyyy MMM d HH:mm:ss.SSS", timezone:"UTC");
set_field("timestamp",time);
end
rule "cisco (4.1) enrich severity"
// this rule will make use of the lookup table
// ciscos-severity-lookuptable
// to transform the severity number into the level name
when
has_field("cisco_message") AND
has_field("severity_level")
then
//mutate simple serverity number to full serverity
let mutateserverity = lookup_value("ciscos-serverity-lookuptable", $message.severity_level);
set_field("severity_level", mutateserverity);
end
rule "cisco (4.2) enrich facility"
// make use of the lookup table
// cisco-ios-facility
// to lookup the facility short codes into
// human readable descriptions
when
has_field("cisco_message") AND
has_field("facility")
then
//translate short facility name to full facility name in extra field
let mutatefacility = lookup_value("cisco-ios-facility", $message.facility);
set_field("facility_description", mutatefacility);
end
Create the pipeline
Now you have the rules, you need to chain them together in a pipeline. Just create a new Pipeline and you might already notice the numbers in the rule names, those are the stages where they need to be placed in.
Create one pipeline with four stages and put the rules according to the numbers into the stages (the first number is the stage). After that is done, you should have one rule in stage one, one rule in stage two, four rules in stage three and two rules in stage four.
As final step - connect that pipeline to a stream that contains messages you want to work on. In our case - the stream where the messages from the RAW input are routed to.
One addition could be to create a fifth stage that contains a rule that routes the messages in a specific stream. But that is up to you. This should be just a starter.
last words
All of the above is needed with Graylog 2.4 - as of the new features in Graylog 3, this above would just be a content pack that includes everything.
I personally did not have any Cisco devices to test the above - so it might be not 100% perfect and you need to adjust the rules. If you need to do that, I would love to get your feedback to include the correct for the next generations.