Working with Cisco ASA / Nexus on Graylog

It is hard to have a working centralized logging environment when you run network devices. Every vendor has his own version and understanding of syslog. Additional most did not speak any kind of structured log format. Some speak some binary format.

With that knowledge and the wish to include Cisco ASA and Nexus devices in your central Graylog, you will need some preparation. Most Important, this need to be done on all devices. But please adjust the settings that need to be changed. The goal is to have messages that have the most value and contain all information that is needed.

For Cisco devices that would look like the following message.

<189>91: *Mar 15 2018 21:48:41.663 UTC: %SYS-5-CONFIG_I: Configured from console by cisco on vty0 (192.168.200.1)

Prep on devices

The steps to get this configured on the Cisco cli, including NTP time sync and the Syslog target set could be similar the next code block. But be aware that you need to adjust the NTP server and the Syslog target to your local needs and requirements. If you change the timezone, you need to adjust settings in the processing in Graylog!

Cisco ASA

This config assumes that all packets are sourced from the management interface of the ASA.

clock timezone UTC 0 0 
no clock summer-time
ntp server 0.0.0.0 prefer source management
logging timestamp
logging trap 6 
logging enable 
logging host management 1.2.3.4 tcp/8514

Cisco Nexus

NX-OS only supports Syslog via UDP! If you have Nexus devices, you need to adjust the processing rules and add a second input in the first rule.

clock timezone UTC 0 0 
no clock summer-time
ntp server 1.3.4.5 prefer use-vrf management
logging timestamp milliseconds # even microseconds possible 
logging server 1.2.3.4 port 8514 use-vrf management
logging source-interface mgmt 0 
logging level all 6
logging origin-id hostname 

Classic IOS

clock timezone UTC 0 0 
no clock summer-time 
ntp server 1.2.3.4 prefer 
ntp update-calendar
service timestamps log datetime msec show-timezone localtime year
logging host 1.2.3.4 transport tcp port 8514 
logging origin-id hostname
logging trap 6

Prep on Graylog

Create a RAW/Plaintext Input in Graylog and get the Input ID. This easiest way would be to select "show received messages" on the input page and copy the ID from the search bar. This will look like 5a71ae996c25ad4b80fbc085 as this is the UUID of the input.

Create in System > Grok Patterns the pattern that is needed in the processing. Click the green Create pattern and add the following.

NAME: CISCOTIMESTAMPTZ 
PATTERN: %{CISCOTIMESTAMP}( %{TZ})?
NAME: NEXUSTIMESTAMP
PATTERN: %{YEAR} %{MONTH} %{MONTHDAY} %{TIME}( %{TZ})?

Create two Lookup Tables in Graylog. Over at System > Lookup Tables create a data adapter ciscos-severity-data and a data adapter cisco-ios-data - the files need to be located on all Graylog servers at the same location. For both need a unique cache - currently I would use a memory cache on a Graylog Node base. If they are named ciscos-severity-cache and cisco-ios-cache it is easy to identify them. Now just create the Lookup Tables, one that is called ciscos-serverity-lookuptable and the other cisco-ios-facility. If the names are changed, the pipeline rules need to be changed too.

The Content for the facility lookup can be found in this gist and here in the same gist. The setup of the Lookup Tables is described in the documentation of Graylog.

Create pipeline rules

We need a few rules that are chained in a pipeline. The first will just identify the Cisco messages - so this is the rule you need to adjust.

rule "cisco (1) flag cisco messages"
when
    // -------------------------------
    // Input MUST be a RAW/Plaintext Input 
    // With a Syslog Input this will not work!
    // -------------------------------
    // The following Configuration for logging of the Cisco Devices
    // is what is expected. All other Configurations might not
    // work
    // -------------------------------
    // #   clock timezone UTC
    // #   no clock summer-time
    // #   ntp server 0.0.0.0 prefer
    // #   ntp server 0.pool.ntp.org
    // #   ntp server 1.pool.ntp.org
    // #   service timestamps log datetime msec show-timezone localtime
    // #   service timestamps debug datetime msec show-timezone localtime
    // #   logging source-interface Loopback0
    // #   logging host 0.0.0.0 transport tcp port 8514
    // #   logging trap 6
    // ----------------------------------------
    // messages will look like
    // <189>91: *Oct 16 2010 21:48:41.663 UTC: %SYS-5-CONFIG_I: Configured from console by cisco on vty0 (192.168.200.1)
    // ----------------------------------------
    // one possible option is when you use a single
    // input for your cisco devices
    // other could be a specific IP range or any other
    // identifier
    // this is only done once - all other rules
    // work with the created field from this rule
    has_field("gl2_source_input") AND
    to_string($message.gl2_source_input) == "5a71ae996c25ad4b80fbc085"
then
    set_field("cisco_message", true);
end
rule "cisco (2) grok extract message"
when
    has_field("cisco_message")
then
     let message_field = to_string($message.message);
    
    // the following GROK Patterns are needed in the System
    // create them in 'System > Grok Patterns' if not present
    // NAME: CISCOTIMESTAMPTZ 
    // PATTERN: %{CISCOTIMESTAMP}( %{TZ})?
    // NAME: NEXUSTIMESTAMP
    // PATTERN: %{YEAR} %{MONTH} %{MONTHDAY} %{TIME}( %{TZ})?
    
    // one of the following pattern will work on IOS messages
    let ios_1 = grok(pattern: "%{SYSLOG5424PRI}(%{NUMBER:log_sequence#})?:( %{NUMBER}:)? %{CISCOTIMESTAMPTZ:log_date}: %%{CISCO_REASON:facility}-%{INT:severity_level}-%{CISCO_REASON:facility_mnemonic}: %{GREEDYDATA:message}", value: message_field, only_named_captures: true);
    let ios_2 = grok(pattern: "%{SYSLOG5424PRI}(%{NUMBER:log_sequence#})?:( %{NUMBER}:)? %{CISCOTIMESTAMPTZ:log_date}: %%{CISCO_REASON:facility}-%{CISCO_REASON:facility_sub}-%{INT:severity_level}-%{CISCO_REASON:facility_mnemonic}: %{GREEDYDATA:message}", value: message_field, only_named_captures: true);

    // one of the following pattern will work on NEXUS messages
    let nexus_1 = grok(pattern: "%{SYSLOG5424PRI}(%{NUMBER:log_sequence#})?: %{NEXUSTIMESTAMP:log_date}: %%{CISCO_REASON:facility}-%{INT:severity_level}-%{CISCO_REASON:facility_mnemonic}: %{GREEDYDATA:message}", value: message_field, only_named_captures: true);
    let nexus_2 = grok(pattern: "%{SYSLOG5424PRI}(%{NUMBER:log_sequence#})?: %{NEXUSTIMESTAMP:log_date}: %%{CISCO_REASON:facility}-%{CISCO_REASON:facility_sub}-%{INT:severity_level}-%{CISCO_REASON:facility_mnemonic}: %{GREEDYDATA:message}", value: message_field, only_named_captures: true);

    // write the fields that are extracted
    set_fields(ios_1);
    set_fields(ios_2);
    set_fields(nexus_1);
    set_fields(nexus_2);
end

rule "cisco (3.1) correct timestamp IOS"
// we want to create ISO8601 Timestamps
// make 'Feb 15 2015 13:33:22.111 UTC' ISO8601
when
    has_field("cisco_message") AND
    has_field("log_date") AND
    grok(pattern: "%{MONTH} %{MONTHDAY} %{YEAR} %{TIME}", value:to_string($message.log_date)).matches == true
then
    let time = parse_date(value:to_string($message.log_date), pattern:"MMM dd yyyy HH:mm:ss.SSS", timezone:"UTC");
    set_field("timestamp",time);
end
rule "cisco (3.2) correct timestamp IOS (single number day)"

// we want to create ISO8601 Timestamps
// make 'Feb  5 2015 13:33:22.111' ISO8601
// cisco did not use 05 but <space>5 for days with a single digit
when
    has_field("cisco_message") AND
    has_field("log_date") AND
    grok(pattern: "%{MONTH}  %{MONTHDAY} %{YEAR} %{TIME}", value:to_string($message.log_date)).matches == true
then

    let time = parse_date(value:to_string($message.log_date), pattern:"MMM  d yyyy HH:mm:ss.SSS", timezone:"UTC");
    set_field("timestamp",time);

end
rule "cisco (3.3) correct timestamp NEXUS"
// we want to create ISO8601 Timestamps
// make '2015 Feb 15 13:33:22.111' ISO8601
when
    has_field("cisco_message") AND
    has_field("log_date") AND
    grok(pattern: "%{YEAR} %{MONTH} %{MONTHDAY} %{TIME}", value:to_string($message.log_date)).matches == true
then
    let time = parse_date(value:to_string($message.log_date), pattern:"yyyy MMM dd HH:mm:ss.SSS", timezone:"UTC");
    set_field("timestamp",time);

end
rule "cisco (3.4) correct timestamp NEXUS (single number day)"
// we want to create ISO8601 Timestamps
// make '2015 Feb  5 13:33:22.111' ISO8601
// cisco did not use 05 but <space>5 for days with a single digit
when
    has_field("cisco_message") AND
    has_field("log_date") AND
    grok(pattern: "%{YEAR} %{MONTH}  %{MONTHDAY} %{TIME}", value:to_string($message.log_date)).matches == true
then
    let time = parse_date(value:to_string($message.log_date), pattern:"yyyy MMM  d HH:mm:ss.SSS", timezone:"UTC");
    set_field("timestamp",time);

end
rule "cisco (4.1) enrich severity"
// this rule will make use of the lookup table 
// ciscos-severity-lookuptable
// to transform the severity number into the level name
when
    has_field("cisco_message") AND
    has_field("severity_level")
then
    //mutate simple serverity number to full serverity
    let mutateserverity = lookup_value("ciscos-serverity-lookuptable", $message.severity_level);
    set_field("severity_level", mutateserverity);
end
rule "cisco (4.2) enrich facility"
// make use of the lookup table
// cisco-ios-facility
// to lookup the facility short codes into 
// human readable descriptions
when
    has_field("cisco_message") AND
    has_field("facility")
then
    //translate short facility name to full facility name in extra field
    let mutatefacility = lookup_value("cisco-ios-facility", $message.facility);
    set_field("facility_description", mutatefacility);
end

Create the pipeline

Now you have the rules, you need to chain them together in a pipeline. Just create a new Pipeline and you might already notice the numbers in the rule names, those are the stages where they need to be placed in.

Create one pipeline with four stages and put the rules according to the numbers into the stages (the first number is the stage). After that is done, you should have one rule in stage one, one rule in stage two, four rules in stage three and two rules in stage four.

As final step - connect that pipeline to a stream that contains messages you want to work on. In our case - the stream where the messages from the RAW input are routed to.

One addition could be to create a fifth stage that contains a rule that routes the messages in a specific stream. But that is up to you. This should be just a starter.

last words

All of the above is needed with Graylog 2.4 - as of the new features in Graylog 3, this above would just be a content pack that includes everything.

I personally did not have any Cisco devices to test the above - so it might be not 100% perfect and you need to adjust the rules. If you need to do that, I would love to get your feedback to include the correct for the next generations.