Add output to Graylog via GELF HTTP input. (#1652)

* Add output to Graylog via GELF HTTP input

docs/graylog/README.rst

@@ -1,26 +1,46 @@
 How to send Cowrie output to Graylog
 ####################################
 
+This guide describes how to configure send cowrie outputs to graylog via syslog and http gelf input.
 
 Prerequisites
-======================
+*************
 
 * Working Cowrie installation
 * Working Graylog installation
 
 Cowrie Configuration
-======================
+********************
+
+Using Syslog
+============
 
 Open the Cowrie configuration file and uncomment these 3 lines::
 
     [output_localsyslog]
-    facility = USER
-    format = text
+    facility * USER
+    format * text
+
+Restart Cowrie
+
+Using GELF HTTP Input
+=====================
+
+Open the Cowrie configuration file and find this block ::
+
+    [output_graylog]
+    enabled * false
+    url * http://127.0.0.1:12201/gelf
+
+Enable this block and specify url of your input.
 
 Restart Cowrie
 
 Graylog Configuration
-======================
+*********************
+
+Syslog Input
+============
 
 Open the Graylog web interface and click on the **System** drop-down in the top menu. From the drop-down menu select **Inputs**. Select **Syslog UDP** from the drop-down menu and click the **Launch new input** button. In the modal dialog enter the following information::
 
@@ -30,8 +50,46 @@ Open the Graylog web interface and click on the **System** drop-down in the top
 
 Then click **Launch.**
 
-Syslog Configuration
-======================
+GELF HTTP Input
+===============
+
+Open the Graylog web interface and click on the **System** drop-down in the top menu. From the drop-down menu select **Inputs**. Select **GELF HTTP** from the drop-down menu and click the **Launch new input** button. In the modal dialog enter the information about your input.
+
+Click **Manage Extractors** near created input. On new page click **Actions** -> **Import extractors**  and paste this config ::
+
+    {
+      "extractors": [
+        {
+          "title": "Cowrie Json Parser",
+          "extractor_type": "json",
+          "converters": [],
+          "order": 0,
+          "cursor_strategy": "copy",
+          "source_field": "message",
+          "target_field": "",
+          "extractor_config": {
+            "list_separator": ", ",
+            "kv_separator": "*",
+            "key_prefix": "",
+            "key_separator": "_",
+            "replace_key_whitespace": false,
+            "key_whitespace_replacement": "_"
+          },
+          "condition_type": "none",
+          "condition_value": ""
+        }
+      ],
+      "version": "4.2.1"
+    }
+
+Then click **Launch.**
+
+Note:
+
+- Do not remove **/gelf** from the end of URL block, expect of case when your proxing this address behind nginx;
+
+Syslog Configuration (For Syslog Output only)
+*********************************************
 
 Create a rsyslog configuration file in /etc/rsyslog.d::
 
@@ -46,4 +104,3 @@ Restart rsyslog::
 
     $ sudo service rsyslog restart
 
-

etc/cowrie.cfg.dist

@@ -775,8 +775,12 @@ enabled = false
 userid = userid_here
 auth_key = auth_key_here
 batch_size = 100
-
-
+#
+# Graylog logging module for GELF http input
+[output_graylog]
+enabled = false
+url = http://graylog.example.com:122011/gelf
+#
 # Local Syslog output module
 #
 # This sends log messages to the local syslog daemon.

src/cowrie/output/graylog.py

@@ -0,0 +1,58 @@
+"""
+Simple Graylog HTTP Graylog Extended Log Format (GELF) logger.
+"""
+
+from __future__ import annotations
+
+import json
+import time
+
+from io import BytesIO
+from twisted.internet import reactor
+from twisted.internet.ssl import ClientContextFactory
+from twisted.web import client, http_headers
+from twisted.web.client import FileBodyProducer
+
+import cowrie.core.output
+from cowrie.core.config import CowrieConfig
+
+
+class Output(cowrie.core.output.Output):
+    def start(self):
+        self.url = CowrieConfig.get("output_graylog", "url").encode("utf8")
+        contextFactory = WebClientContextFactory()
+        self.agent = client.Agent(reactor, contextFactory)
+
+    def stop(self):
+        pass
+
+    def write(self, logentry):
+        for i in list(logentry.keys()):
+            # Remove twisted 15 legacy keys
+            if i.startswith("log_"):
+                del logentry[i]
+
+        gelf_message = {
+            'version': '1.1',
+            'host': logentry["sensor"],
+            'timestamp': time.time(),
+            'short_message': json.dumps(logentry),
+            'level': 1,
+        }
+
+        self.postentry(gelf_message)
+
+    def postentry(self, entry):
+        headers = http_headers.Headers(
+            {
+                b"Content-Type": [b"application/json"],
+            }
+        )
+
+        body = FileBodyProducer(BytesIO(json.dumps(entry).encode("utf8")))
+        self.agent.request(b"POST", self.url, headers, body)
+
+
+class WebClientContextFactory(ClientContextFactory):
+    def getContext(self, hostname, port):
+        return ClientContextFactory.getContext(self)