listbot integration

Update install.sh

.github/ISSUE_TEMPLATE.md

@@ -1,31 +1,29 @@
-# Contribution
+# Issues
 
-Thank you for your decision to contribute to T-Pot.
-
-## Issues
-
-Please feel free to post your problems, ideas and issues [here](https://github.com/dtag-dev-sec/tpotce/issues). We will try to answer ASAP, but to speed things up we encourage you to ...
-- [ ] Use the [search function](https://github.com/dtag-dev-sec/tpotce/issues?utf8=%E2%9C%93&q=) first
-- [ ] Check the FAQs in our [WIKI](https://github.com/dtag-dev-sec/tpotce/wiki)
-- [ ] Provide [basic support information](#info) with regard to your issue
-
-Thank you :smiley:
-
-
--
+Before you post your issue make sure it has not been answered yet and provide `basic support information` if you come to the conclusion it is a new issue:
+- 🔍 Use the [search function](https://github.com/dtag-dev-sec/tpotce/issues?utf8=%E2%9C%93&q=) first
+- 🧐 Check our [WIKI](https://github.com/dtag-dev-sec/tpotce/wiki)
+- 📚 Consult the documentation of 💻 [Debian](https://www.debian.org/doc/), 🐳 [Docker](https://docs.docker.com/), the 🦌 [ELK stack](https://www.elastic.co/guide/index.html) and the 🍯 [T-Pot Readme](https://github.com/dtag-dev-sec/tpotce/blob/master/README.md).
+- **⚠️ Provide [basic support information](#info) or similiar information with regard to your issue or we can not help you and will close the issue without further notice**
 
+<br>
+<br>
+<br>
 
 <a name="info"></a>
-### Basic support information
+## ⚠️ Basic support information (commands are expected to run as `root`)
 
+- What version of the OS are you currently using `lsb_release -a` and `uname -a`?
 - What T-Pot version are you currently using?
-- Are you running on a Intel NUC or a VM?
+- What edition (Standard, Nextgen, etc.) of T-Pot are you running?
+- What architecture are you running on (i.e. hardware, cloud, VM, etc.)?
+- Did you have any problems during the install? If yes, please attach `/install.log` `/install.err`.
 - How long has your installation been running?
-- Did you install any upgrades or packages?
-- Did you modify any scripts?
-- Have you turned persistence on/off?
-- How much RAM is available (login via ssh and run `htop`)?
-- How much stress are the CPUs under (login via ssh and run `htop`)?
-- How much swap space is being used (login via ssh and run `htop`)?
-- How much free disk space is available (login via ssh and run `sudo df -h`)?
-- What is the current container status (login via ssh and run `sudo dps.sh`)?
+- Did you install upgrades, packages or use the update script?
+- Did you modify any scripts or configs? If yes, please attach the changes.
+- Please provide a screenshot of `glances` and `htop`.
+- How much free disk space is available (`df -h`)?
+- What is the current container status (`dps.sh`)?
+- What is the status of the T-Pot service (`systemctl status tpot`)?
+- What ports are being occupied? Stop T-Pot `systemctl stop tpot` and run `netstat -tulpen`
+- If a single container shows as `DOWN` you can run `docker logs <container-name>` for the latest log entries

CHANGELOG.md

@@ -0,0 +1,188 @@
+# Changelog
+
+## 20200116
+- **Bump ELK to latest 6.8.6**
+- **Update ISO image to fix upstream bug of missing kernel modules**
+- **Include dashboards for CitrixHoneypot**
+  - Please run `/opt/tpot/update.sh` for the necessary modifications, omit the reboot and run `/opt/tpot/bin/tped.sh` to (re-)select the NextGen installation type.
+  - This update requires the latest Kibana objects as well. Download the latest from https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/etc/objects/kibana_export.json.zip, unzip and import the objects within Kibana WebUI > Management > Saved Objects > Export / Import". All objects will be overwritten upon import, make sure to run an export first.
+
+## 20200115
+- **Prepare integration of CitrixHoneypot**
+  - Prepare integration of [CitrixHoneypot](https://github.com/MalwareTech/CitrixHoneypot) by MalwareTech
+  - Integration into ELK is still open
+  - Please run `/opt/tpot/update.sh` for the necessary modifications, omit the reboot and run `/opt/tpot/bin/tped.sh` to (re-)select the NextGen installation type.
+
+## 20191224
+- **Use pigz, optimize logrotate.conf**
+  - Use `pigz` for faster archiving, especially with regard to high volumes of logs - Thanks to @workandresearchgithub!
+  - Optimize `logrotate.conf` to improve archiving speed and get rid of multiple compression, also introduce `pigz`.
+
+## 20191121
+- **Bump ADBHoney to latest master**
+  - Use latest version of ADBHoney, which now fully support Python 3.x - Thanks to @huuck!
+
+## 20191113, 20191104, 20191103, 20191028
+- **Switch to Debian 10 on OTC, Ansible Improvements**
+  - OTC now supporting Debian 10 - Thanks to @shaderecker!
+
+## 20191028
+- **Fix an issue with pip3, yq**
+  - `yq` needs rehashing.
+
+## 20191026
+- **Remove cockpit-pcp**
+  - `cockpit-pcp` floods swap for some reason - removing for now.
+
+## 20191022
+- **Bump Suricata to 5.0.0**
+
+## 20191021
+- **Bump Cowrie to 2.0.0**
+
+## 20191016
+- **Tweak installer, pip3, Heralding**
+  - Install `cockpit-pcp` right from the start for machine monitoring in cockpit.
+  - Move installer and update script to use pip3.
+  - Bump heralding to latest master (1.0.6) - Thanks @johnnykv!
+
+## 20191015
+- **Tweaking, Bump glutton, unlock ES script**
+  - Add `unlock.sh` to unlock ES indices in case of lockdown after disk quota has been reached.
+  - Prevent too much terminal logging from p0f and glutton since `daemon.log` was filled up.
+  - Bump glutton to latest master now supporting payload_hex. Thanks to @glaslos.
+
+## 20191002
+- **Merge**
+  - Support Debian Buster images for AWS #454
+  - Thank you @piffey
+
+## 20190924
+- **Bump EWSPoster**
+  - Supports Python 3.x
+  - Thank you @Trixam
+
+## 20190919
+- **Merge**
+  - Handle non-interactive shells #454
+  - Thank you @Oogy
+
+## 20190907
+- **Logo tweaking**
+  - Add QR logo
+
+## 20190828
+- **Upgrades and rebuilds**
+  - Bump Medpot, Nginx and Adbhoney to latest master
+  - Bump ELK stack to 6.8.2
+  - Rebuild Mailoney, Honeytrap, Elasticpot and Ciscoasa
+  - Add 1080p T-Pot wallpaper for download
+
+## 20190824
+- **Add some logo work**
+  - Thanks to @thehadilps's suggestion adjusted social preview
+  - Added 4k T-Pot wallpaper for download
+
+## 20190823
+- **Fix for broken Fuse package**
+  - Fuse package in upstream is broken
+  - Adjust installer as workaround, fixes #442
+
+## 20190816
+- **Upgrades and rebuilds**
+  - Adjust Dionaea to avoid nmap detection, fixes #435 (thanks @iukea1)
+  - Bump Tanner, Cyberchef, Spiderfoot and ES Head to latest master
+
+## 20190815
+- **Bump ELK stack to 6.7.2**
+  - Transition to 7.x must iterate slowly through previous versions to prevent changes breaking T-Pots
+
+## 20190814
+- **Logstash Translation Maps improvement**
+  - Download translation maps rather than running a git pull
+  - Translation maps will now be bzip2 compressed to reduce traffic to a minimum
+  - Fixes #432
+
+## 20190802
+- **Add support for Buster as base image**
+  - Install ISO is now based on Debian Buster
+  - Installation upon Debian Buster is now supported
+
+## 20190701
+- **Reworked Ansible T-Pot Deployment**
+  - Transitioned from bash script to all Ansible
+  - Reusable Ansible Playbook for OpenStack clouds
+  - Example Showcase with our Open Telekom Cloud
+  - Adaptable for other cloud providers
+
+## 20190626
+- **HPFEEDS Opt-In commandline option**
+  - Pass a hpfeeds config file as a commandline argument
+  - hpfeeds config is saved in `/data/ews/conf/hpfeeds.cfg`
+  - Update script restores hpfeeds config
+
+## 20190604
+- **Finalize Fatt support**
+  - Build visualizations, searches, dashboards
+  - Rebuild index patterns
+  - Some finishing touches
+
+## 20190601
+- **Start supporting Fatt, remove Glastopf**
+  - Build Dockerfile, Adjust logstash, installer, update and such.
+  - Glastopf is no longer supported within T-Pot
+
+## 20190528+20190531
+- **Increase total number of fields**
+  - Adjust total number of fileds for logstash templae from 1000 to 2000.
+
+## 20190526
+- **Fix build for Cowrie**
+  - Upstream changes required a new package `py-bcrypt`.
+
+## 20190525
+- **Fix build for RDPY**
+  - Building was prevented due to cache error which occurs lately on Alpine if `apk` is using `--no-ache' as options.
+
+## 20190520
+- **Adjust permissions for /data folder**
+  - Now it is possible to download files from `/data` using SCP, WINSCP or CyberDuck.
+
+## 20190513
+- **Added Ansible T-Pot Deployment on Open Telekom Cloud**
+  - Reusable Ansible Playbooks for all cloud providers
+  - Example Showcase with our Open Telekom Cloud
+
+## 20190511
+- **Add hptest script**
+  - Quickly test if the honeypots are working with `hptest.sh <[ip,host]>` based on nmap.
+
+## 20190508
+- **Add tsec / install user to tpot group**
+  - For users being able to easily download logs from the /data folder the installer now adds the `tpot` or the logged in user (`who am i`) via `usermod -a -G tpot <user>` to the tpot group. Also /data permissions will now be enforced to `770`, which is necessary for directory listings.
+
+## 20190502
+- **Fix KVPs**
+  - Some KVPs for Cowrie changed and the tagcloud was not showing any values in the Cowrie dashboard.
+  - New installations are not affected, however existing installations need to import the objects from /opt/tpot/etc/objects/kibana-objects.json.zip.
+- **Makeiso**
+  - Move to Xorriso for building the ISO image.
+  - This allows to support most of the Debian based distros, i.e. Debian, MxLinux and Ubuntu.
+
+## 20190428
+- **Rebuild ISO**
+  - The install ISO needed a rebuilt after some changes in the Debian mirrors.
+- **Disable Netselect**
+  - After some reports in the issues that some Debian mirrors were not fully synced and thus some packages were unavailable the netselect-apt feature was disabled.
+
+## 20190406
+- **Fix for SSH**
+  - In some situations the SSH Port was not written to a new line (thanks to @dpisano for reporting).
+- **Fix race condition for apt-fast**
+  - Curl and wget need to be installed before apt-fast installation.
+
+## 20190404
+- **Fix #332**
+  - If T-Pot, opposed to the requirements, does not have full internet access netselect-apt fails to determine the fastest mirror as it needs ICMP and UDP outgoing. Should netselect-apt fail the default mirrors will be used.
+- **Improve install speed with apt-fast**
+  - Migrating from a stable base install to Debian (Sid) requires downloading lots of packages. Depending on your geo location the download speed was already improved by introducing netselect-apt to determine the fastest mirror. With apt-fast the downloads will be even faster by downloading packages not only in parallel but also with multiple connections per package.

CONTRIBUTING.MD

@@ -1,31 +0,0 @@
-# Contribution
-
-Thank you for your decision to contribute to T-Pot.
-
-## Issues
-
-Please feel free to post your problems, ideas and issues [here](https://github.com/dtag-dev-sec/tpotce/issues). We will try to answer ASAP, but to speed things up we encourage you to ...
-- [ ] Use the [search function](https://github.com/dtag-dev-sec/tpotce/issues?utf8=%E2%9C%93&q=) first
-- [ ] Check the FAQs in our [WIKI](https://github.com/dtag-dev-sec/tpotce/wiki)
-- [ ] Provide [basic support information](#info) with regard to your issue
-
-Thank you :smiley:
-
-
--
-
-
-<a name="info"></a>
-### Basic support information
-
-- What T-Pot version are you currently using?
-- Are you running on a Intel NUC or a VM?
-- How long has your installation been running?
-- Did you install any upgrades or packages?
-- Did you modify any scripts?
-- Have you turned persistence on/off?
-- How much RAM is available (login via ssh and run `htop`)?
-- How much stress are the CPUs under (login via ssh and run `htop`)?
-- How much swap space is being used (login via ssh and run `htop`)?
-- How much free disk space is available (login via ssh and run `sudo df -h`)?
-- What is the current container status (login via ssh and run `sudo dps.sh`)?

README.md

@@ -1,6 +1,6 @@
-# T-Pot 18.11
+![T-Pot](doc/tpotsocial.png)
 
-T-Pot 18.11 runs on the latest 18.04.x LTS Ubuntu Server Network Installer image, is based on
+T-Pot 19.03 runs on Debian (Sid), is based heavily on
 
 [docker](https://www.docker.com/), [docker-compose](https://docs.docker.com/compose/)
 
@@ -8,13 +8,14 @@ and includes dockerized versions of the following honeypots
 
 * [adbhoney](https://github.com/huuck/ADBHoney),
 * [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot),
+* [citrixhoneypot](https://github.com/MalwareTech/CitrixHoneypot),
 * [conpot](http://conpot.org/),
-* [cowrie](http://www.micheloosterhof.com/cowrie/),
+* [cowrie](https://github.com/cowrie/cowrie),
 * [dionaea](https://github.com/DinoTools/dionaea),
-* [elasticpot](https://github.com/schmalle/ElasticPot),
-* [glastopf](http://mushmush.org/),
+* [elasticpot](https://github.com/schmalle/ElasticpotPY),
 * [glutton](https://github.com/mushorg/glutton),
 * [heralding](https://github.com/johnnykv/heralding),
+* [honeypy](https://github.com/foospidy/HoneyPy),
 * [honeytrap](https://github.com/armedpot/honeytrap/),
 * [mailoney](https://github.com/awhitehatter/mailoney),
 * [medpot](https://github.com/schmalle/medpot),
@@ -29,6 +30,7 @@ Furthermore we use the following tools
 * [Cyberchef](https://gchq.github.io/CyberChef/) a web app for encryption, encoding, compression and data analysis.
 * [ELK stack](https://www.elastic.co/videos) to beautifully visualize all the events captured by T-Pot.
 * [Elasticsearch Head](https://mobz.github.io/elasticsearch-head/) a web front end for browsing and interacting with an Elastic Search cluster.
+* [Fatt](https://github.com/0x4D31/fatt) a pyshark based script for extracting network metadata and fingerprints from pcap files and live network traffic.
 * [Spiderfoot](https://github.com/smicallef/spiderfoot) a open source intelligence automation tool.
 * [Suricata](http://suricata-ids.org/) a Network Security Monitoring engine.
 
@@ -51,6 +53,9 @@ Furthermore we use the following tools
   - [Running on Hardware](#hardware)
   - [Post Install User](#postinstall)
   - [Post Install Auto](#postinstallauto)
+  - [Cloud Deployments](#cloud)
+    - [Ansible](#ansible)
+    - [Terraform](#terraform)
   - [First Run](#firstrun)
   - [System Placement](#placement)
 - [Updates](#updates)
@@ -60,6 +65,7 @@ Furthermore we use the following tools
   - [Tools](#tools)
   - [Maintenance](#maintenance)
   - [Community Data Submission](#submission)
+  - [Opt-In HPFEEDS Data Submission](#hpfeeds-optin)
 - [Roadmap](#roadmap)
 - [Disclaimer](#disclaimer)
 - [FAQ](#faq)
@@ -67,71 +73,81 @@ Furthermore we use the following tools
 - [Licenses](#licenses)
 - [Credits](#credits)
 - [Stay tuned](#staytuned)
+- [Testimonial](#testimonial)
 - [Fun Fact](#funfact)
 
 <a name="changelog"></a>
-# Changelog
-- **New honeypots**
-  - *Adbhoney* Low interaction honeypot designed for Android Debug Bridge over TCP/IP.
-  - *Ciscoasa* a low interaction honeypot for the Cisco ASA component capable of detecting CVE-2018-0101, a DoS and remote code execution vulnerability.
-  - *Glutton* (NextGen) is the all eating honeypot
-  - *Heralding* a credentials catching honeypot.
-  - *Medpot* is a HL7 / FHIR honeypot.
-  - *Snare* is a web application honeypot sensor, is the successor of Glastopf. SNARE has feature parity with Glastopf and allows to convert existing web pages into attack surfaces.
-  - *Tanner* is SNARES' "brain". Every event is send from SNARE to TANNER, gets evaluated and TANNER decides how SNARE should respond to the client. This allows us to change the behaviour of many sensors on the fly. We are providing a TANNER instance for your use, but there is nothing stopping you from setting up your own instance.
-- **New tools**
-  - *Cockpit* is an interactive server admin interface. It is easy to use and very lightweight. Cockpit interacts directly with the operating system from a real Linux session in a browser.
-  - *Cyberchef* is the Cyber Swiss Army Knife - a web app for encryption, encoding, compression and data analysis.
-  - *grc* (commandline) is yet another colouriser (written in python) for beautifying your logfiles or output of commands.
-  - *multitail* (commandline) allows you to monitor logfiles and command output in multiple windows in a terminal, colorize, filter and merge.
-  - *tped.sh* (commandline) allows you to switch between T-Pot Editions after installation.
-- **Deprecated tools**
-  - *Netdata*, *Portainer* and *WeTTY* were superseded by *Cockpit* which is much more lightweight, perfectly well integrated into Ubuntu 18.04 LTS and of course comes with the same but a more basic feature set.
-- **New Standard Installation**
-  - The new standard installation is now running a whopping *14* honeypot instances.
-- **T-Pot Universal Installer**
-  - The T-Pot installer now also includes the option to install on a existing machine, the T-Pot-Autoinstaller is no longer necessary.
-- **Tighten Security**
-  - The docker containers are now running mostly with a read-only file system
-  - If possible using `setcap` to start daemons without root or dropping privileges
-  - Introducing `fail2ban` to ease up on `authorized_keys` requirement which is no longer necessary for `SSH`. Also to further prevent brute-force attacks on `Cockpit` and `NGINX` allowing for faster load times of the WebUI.
-- **Iptables exceptions for NFQ based honeypots**
-  - In previous versions `iptables`had manually be maintained, now a a script parses `/opt/tpot/etc/tpot.yml` and extracts port information to automatically generate exceptions for ports that should not be forwarded to NFQ.
-- **CI**
-  - The Kibana UI now uses a magenta theme.
-- **ES HEAD**
-  - A Java Script now automatically enters the correct FQDN / IP. A manual step is no longer required.
-- **ELK STACK**
-  - The ELK Stack was updated to the latest 6.x versions.
-  - This also means you can now expect the availability of basic *X-Pack-Feaures*, the full feature set however is only available to users with a valid license.
-- **Dashboards Makeover**
-  - Because Kibana 6.x introduced so much whitespace the dashboards and some of the visualizations needed some overhaul. While it probably needs some getting used to the key was to focus on displaying as much information while not compromising on clarity.  
-  - Because of the new honeypots we now more than **200 Visualizations** pre-configured and compiled to 16 individual **Kibana Dashboards**. Monitor all *honeypot events* locally on your T-Pot installation. Aside from *honeypot events* you can also view *Suricata NSM and NGINX* events for a quick overview of wire events.
-- **Honeypot updates and improvements**
-  - All honeypots were updated to their latest stable versions.
-  - Docker images were mostly overhauled to tighten security even further
-  - Some of the honeypot configurations were modified to keep things fresh
+# Release Notes
+- **Move from Ubuntu 18.04 to Debian (Sid)**
+  - For almost 5 years Ubuntu LTS versions were our distributions of choice. Last year we made a design choice for T-Pot to be closer to a rolling release model and thus allowing us to issue smaller changes and releases in a more timely manner. The distribution of choice is Debian (Sid / unstable) which will provide us with the latest advancements in a Debian based distribution.
+- **Include HoneyPy honeypot**
+  - *HoneyPy* is now included in the NEXTGEN installation type
+- **Include Suricata 4.1.3**
+  - Building *Suricata 4.1.3* from scratch to enable JA3 and overall better protocol support.
+- **Update tools to the latest versions**
+  - ELK Stack 6.6.2
+  - CyberChef 8.27.0
+  - SpiderFoot v3.0
+  - Cockpit 188
+  - NGINX is now built to enforce TLS 1.3 on the T-Pot WebUI
+- **Update honeypots**
+  - Where possible / feasible the honeypots have been updated to their latest versions.
+  - *Cowrie* now supports *HASSH* generated hashes which allows for an easier identification of an attacker accross IP adresses.
+  - *Heralding* now supports *SOCKS5* emulation.
+- **Update Dashboards & Visualizations**
+  - *Offset Dashboard* added to easily spot changes in attacks on a single dashboard in 24h time window.
+  - *Cowrie Dashboard* modified to integrate *HASSH* support / visualizations.
+  - *HoneyPy Dashboard* added to support latest honeypot addition.
+  - *Suricata Dashboard* modified to integrate *JA3* support / visualizations.
+- **Debian mirror selection**
+  - During base install you now have to manually select a mirror.
+  - Upon T-Pot install the mirror closest to you will be determined automatically, `netselect-apt` requires you to allow ICMP outbound.
+  - This solves peering problems for most of the users speeding up installation and updates.
+- **Bugs**
+  - Fixed issue #298 where the import and export of objects on the shell did not work.
+  - Fixed issue #313 where Spiderfoot raised a KeyError, which was previously fixed in upstream.
+  - Fixed error in Suricata where path for reference.config changed.
+- **Release Cycle**
+  - As far as possible we will integrate changes now faster into the master branch, eliminating the need for monolithic releases. The update feature will be continuously improved on that behalf. However this might not account for all feature changes.
+- **HPFEEDS Opt-In**
+  - If you want to share your T-Pot data with a 3rd party HPFEEDS broker such as [SISSDEN](https://sissden.eu) you can do so by creating an account at the SISSDEN portal and run `hpfeeds_optin.sh` on T-Pot.
 - **Update Feature**
-  - For the ones who like to live on the bleeding edge of T-Pot development there is now a update script available in `/opt/tpot/update.sh`.
-  - This feature is now in beta and is mostly intended to provide you with the latest development advances without the need of reinstalling T-Pot.
+  - For the ones who like to live on the bleeding edge of T-Pot development there is now an update script available in `/opt/tpot/update.sh`.
+  - This feature is beta and is mostly intended to provide you with the latest development advances without the need of reinstalling T-Pot.
+- **Deprecated tools**
+  - *ctop* will no longer be part of T-Pot.
+- **Fix #332**
+  - If T-Pot, opposed to the requirements, does not have full internet access netselect-apt fails to determine the fastest mirror as it needs ICMP and UDP outgoing. Should netselect-apt fail the default mirrors will be used.
+- **Improve install speed with apt-fast**
+  - Migrating from a stable base install to Debian (Sid) requires downloading lots of packages. Depending on your geo location the download speed was already improved by introducing netselect-apt to determine the fastest mirror. With apt-fast the downloads will be even faster by downloading packages not only in parallel but also with multiple connections per package.
+- **HPFEEDS Opt-In commandline option**
+  - Pass a hpfeeds config file as a commandline argument
+  - hpfeeds config is saved in `/data/ews/conf/hpfeeds.cfg`
+  - Update script restores hpfeeds config
+- **Ansible T-Pot Deployment**
+  - Transitioned from bash script to all Ansible
+  - Reusable Ansible Playbook for OpenStack clouds
+  - Example Showcase with our Open Telekom Cloud
+  - Adaptable for other cloud providers
 
 <a name="concept"></a>
 # Technical Concept
 
-T-Pot is based on the network installer of Ubuntu Server 18.04.x LTS.
+T-Pot is based on the network installer Debian (Stretch). During installation the whole system will be updated to Debian (Sid).
 The honeypot daemons as well as other support components being used have been containerized using [docker](http://docker.io).
 This allows us to run multiple honeypot daemons on the same network interface while maintaining a small footprint and constrain each honeypot within its own environment.
 
 In T-Pot we combine the dockerized honeypots ...
 * [adbhoney](https://github.com/huuck/ADBHoney),
 * [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot),
+* [citrixhoneypot](https://github.com/MalwareTech/CitrixHoneypot),
 * [conpot](http://conpot.org/),
 * [cowrie](http://www.micheloosterhof.com/cowrie/),
 * [dionaea](https://github.com/DinoTools/dionaea),
-* [elasticpot](https://github.com/schmalle/ElasticPot),
-* [glastopf](http://mushmush.org/),
+* [elasticpot](https://github.com/schmalle/ElasticpotPY),
 * [glutton](https://github.com/mushorg/glutton),
 * [heralding](https://github.com/johnnykv/heralding),
+* [honeypy](https://github.com/foospidy/HoneyPy),
 * [honeytrap](https://github.com/armedpot/honeytrap/),
 * [mailoney](https://github.com/awhitehatter/mailoney),
 * [medpot](https://github.com/schmalle/medpot),
@@ -144,14 +160,15 @@ In T-Pot we combine the dockerized honeypots ...
 * [Cyberchef](https://gchq.github.io/CyberChef/) a web app for encryption, encoding, compression and data analysis.
 * [ELK stack](https://www.elastic.co/videos) to beautifully visualize all the events captured by T-Pot.
 * [Elasticsearch Head](https://mobz.github.io/elasticsearch-head/) a web front end for browsing and interacting with an Elastic Search cluster.
+* [Fatt](https://github.com/0x4D31/fatt) a pyshark based script for extracting network metadata and fingerprints from pcap files and live network traffic.
 * [Spiderfoot](https://github.com/smicallef/spiderfoot) a open source intelligence automation tool.
 * [Suricata](http://suricata-ids.org/) a Network Security Monitoring engine.
 
-... to give you the best out-of-the-box experience possible and a easy-to-use multi-honeypot appliance.
+... to give you the best out-of-the-box experience possible and an easy-to-use multi-honeypot appliance.
 
 ![Architecture](doc/architecture.png)
 
-While data within docker containers is volatile we do now ensure a default 30 day persistence of all relevant honeypot and tool data in the well known `/data` folder and sub-folders. The persistence configuration may be adjusted in `/opt/tpot/etc/logrotate/logrotate.conf`. Once a docker container crashes, all other data produced within its environment is erased and a fresh instance is started from the corresponding docker image.<br>
+While data within docker containers is volatile we do ensure a default 30 day persistence of all relevant honeypot and tool data in the well known `/data` folder and sub-folders. The persistence configuration may be adjusted in `/opt/tpot/etc/logrotate/logrotate.conf`. Once a docker container crashes, all other data produced within its environment is erased and a fresh instance is started from the corresponding docker image.<br>
 
 Basically, what happens when the system is booted up is the following:
 
@@ -170,7 +187,7 @@ The individual docker configurations are located in the [docker folder](https://
 Depending on your installation type, whether you install on [real hardware](#hardware) or in a [virtual machine](#vm), make sure your designated T-Pot system meets the following requirements:
 
 ##### Standard Installation
-- Honeypots: adbhoney, ciscoasa, conpot, cowrie, dionaea, elasticpot, heralding, honeytrap, mailoney, rdpy, snare, tanner and vnclowpot
+- Honeypots: adbhoney, ciscoasa, conpot, cowrie, dionaea, elasticpot, heralding, honeytrap, mailoney, medpot, rdpy, snare & tanner
 - Tools: cockpit, cyberchef, ELK, elasticsearch head, ewsposter, NGINX, spiderfoot, p0f and suricata
 
 - 6-8 GB RAM (less RAM is possible but might introduce swapping)
@@ -179,7 +196,7 @@ Depending on your installation type, whether you install on [real hardware](#har
 - A working, non-proxied, internet connection
 
 ##### Sensor Installation
-- Honeypots: adbhoney, ciscoasa, conpot, cowrie, dionaea, elasticpot, heralding, honeytrap, mailoney, rdpy, snare, tanner and vnclowpot
+- Honeypots: adbhoney, ciscoasa, conpot, cowrie, dionaea, elasticpot, heralding, honeytrap, mailoney, medpot, rdpy, snare & tanner
 - Tools: cockpit
 
 - 6-8 GB RAM (less RAM is possible but might introduce swapping)
@@ -188,7 +205,7 @@ Depending on your installation type, whether you install on [real hardware](#har
 - A working, non-proxied, internet connection
 
 ##### Industrial Installation
-- Honeypots: conpot, rdpy, vnclowpot
+- Honeypots: conpot, cowrie, heralding, medpot, rdpy
 - Tools: cockpit, cyberchef, ELK, elasticsearch head, ewsposter, NGINX, spiderfoot, p0f and suricata
 
 - 6-8 GB RAM (less RAM is possible but might introduce swapping)
@@ -205,18 +222,9 @@ Depending on your installation type, whether you install on [real hardware](#har
 - Network via DHCP
 - A working, non-proxied, internet connection
 
-##### NextGen Installation (Glutton instead of Honeytrap)
-- Honeypots: adbhoney, ciscoasa, conpot, cowrie, dionaea, elasticpot, glutton, heralding, mailoney, rdpy, snare, tanner and vnclowpot
-- Tools: cockpit, cyberchef, ELK, elasticsearch head, ewsposter, NGINX, spiderfoot, p0f and suricata
-
-- 6-8 GB RAM (less RAM is possible but might introduce swapping)
-- 128 GB SSD (smaller is possible but limits the capacity of storing events)
-- Network via DHCP
-- A working, non-proxied, internet connection
-
-##### Legacy Installation (honeypots based on Standard Installation of T-Pot 17.10)
-- Honeypots: cowrie, dionaea, elasticpot, glastopf, honeytrap, mailoney, rdpy and vnclowpot
-- Tools: cockpit, cyberchef, ELK, elasticsearch head, ewsposter, NGINX, spiderfoot, p0f and suricata
+##### NextGen Installation (Glutton replacing Honeytrap, HoneyPy replacing Elasticpot)
+- Honeypots: adbhoney, ciscoasa, citrixhoneypot, conpot, cowrie, dionaea, glutton, heralding, honeypy, mailoney, rdpy, snare & tanner
+- Tools: cockpit, cyberchef, ELK, elasticsearch head, ewsposter, fatt, NGINX, spiderfoot, p0f and suricata
 
 - 6-8 GB RAM (less RAM is possible but might introduce swapping)
 - 128 GB SSD (smaller is possible but limits the capacity of storing events)
@@ -227,7 +235,7 @@ Depending on your installation type, whether you install on [real hardware](#har
 # Installation
 The installation of T-Pot is straight forward and heavily depends on a working, transparent and non-proxied up and running internet connection. Otherwise the installation **will fail!**
 
-Firstly, decide if you want to download our prebuilt installation ISO image from [GitHub](https://github.com/dtag-dev-sec/tpotce/releases), [create it yourself](#createiso) ***or*** [post-install on a existing Ubuntu Server 18.04 LTS](#postinstall).
+Firstly, decide if you want to download our prebuilt installation ISO image from [GitHub](https://github.com/dtag-dev-sec/tpotce/releases), [create it yourself](#createiso) ***or*** [post-install on an existing Debian 9.7 (Stretch)](#postinstall).
 
 Secondly, decide where you want to let the system run: [real hardware](#hardware) or in a [virtual machine](#vm)?
 
@@ -241,7 +249,7 @@ You can download the prebuilt installation image from [GitHub](https://github.co
 For transparency reasons and to give you the ability to customize your install, we provide you the [ISO Creator](https://github.com/dtag-dev-sec/tpotce) that enables you to create your own ISO installation image.
 
 **Requirements to create the ISO image:**
-- Ubuntu 18.04 LTS or newer as host system (others *may* work, but *remain* untested)
+- Debian 9.7 or newer as host system (others *may* work, but *remain* untested)
 - 4GB of free memory  
 - 32GB of free storage
 - A working internet connection
@@ -284,17 +292,17 @@ If you decide to run T-Pot on dedicated hardware, just follow these steps:
 Whereas most CD burning tools allow you to burn from ISO images, the procedure to create a bootable USB stick from an ISO image depends on your system. There are various Windows GUI tools available, e.g. [this tip](http://www.ubuntu.com/download/desktop/create-a-usb-stick-on-windows) might help you.<br> On [Linux](http://askubuntu.com/questions/59551/how-to-burn-a-iso-to-a-usb-device) or [MacOS](http://www.ubuntu.com/download/desktop/create-a-usb-stick-on-mac-osx) you can use the tool *dd* or create the USB stick with T-Pot's [ISO Creator](https://github.com/dtag-dev-sec).
 2. Boot from the USB stick and install.
 
-*Please note*: We will ensure the compatibility with the Intel NUC platform, as we really like the form factor, looks and build quality. Other platforms **remain untested**.
+*Please note*: While we are performing limited tests with the Intel NUC platform other hardware platforms **remain untested**. We can not provide hardware support of any kind.
 
 <a name="postinstall"></a>
 ## Post-Install User
-In some cases it is necessary to install Ubuntu Server 18.04 LTS on your own:
+In some cases it is necessary to install Debian 9.7 (Stretch) on your own:
  - Cloud provider does not offer mounting ISO images.
  - Hardware setup needs special drivers and / or kernels.
  - Within your company you have to setup special policies, software etc.
  - You just like to stay on top of things.
 
-While the T-Pot-Autoinstaller served us perfectly well in the past we decided to include the feature directly into T-Pot and its Universal Installer.
+The T-Pot Universal Installer will upgrade the system to Debian (Sid) and install all required T-Pot dependencies.
 
 Just follow these steps:
 
@@ -308,7 +316,7 @@ The installer will now start and guide you through the install process.
 
 <a name="postinstallauto"></a>
 ## Post-Install Auto
-You can also let the installer run automatically if you provide your own `tpot.conf`. A example is available in `tpotce/iso/installer/tpot.conf.dist`. This should make things easier in case you want to automate the installation i.e. with **Ansible**.
+You can also let the installer run automatically if you provide your own `tpot.conf`. An example is available in `tpotce/iso/installer/tpot.conf.dist`. This should make things easier in case you want to automate the installation i.e. with **Ansible**.
 
 Just follow these steps while adjusting `tpot.conf` to your needs:
 
@@ -321,6 +329,30 @@ cp tpot.conf.dist tpot.conf
 
 The installer will start automatically and guide you through the install process.
 
+<a name="cloud"></a>
+## Cloud Deployments
+Located in the [`cloud`](cloud) folder.  
+Currently there are examples with Ansible & Terraform.  
+If you would like to contribute, you can add other cloud deployments like Chef or Puppet or extend current methods with other cloud providers.
+
+<a name="ansible"></a>
+### Ansible Deployment
+You can find an [Ansible](https://www.ansible.com/) based T-Pot deployment in the [`cloud/ansible`](cloud/ansible) folder.  
+The Playbook in the [`cloud/ansible/openstack`](cloud/ansible/openstack) folder is reusable for all OpenStack clouds out of the box.
+
+It first creates all resources (security group, network, subnet, router), deploys a new server and then installs and configures T-Pot.
+
+You can have a look at the Playbook and easily adapt the deploy role for other [cloud providers](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html).
+
+<a name="terraform"></a>
+### Terraform Configuration
+
+You can find [Terraform](https://www.terraform.io/) configuration in the [`cloud/terraform`](cloud/terraform) folder.
+
+This can be used to launch a virtual machine, bootstrap any dependencies and install T-Pot in a single step.
+
+Configuration for Amazon Web Services (AWS) is currently included and this can easily be extended to support other [Terraform providers](https://www.terraform.io/docs/providers/index.html).
+
 <a name="firstrun"></a>
 ## First Run
 The installation requires very little interaction, only a locale and keyboard setting have to be answered for the basic linux installation. The system will reboot and please maintain the active internet connection. The T-Pot installer will start and ask you for an installation type, password for the **tsec** user and credentials for a **web user**. Everything else will be configured automatically. All docker images and other componenents will be downloaded. Depending on your network connection and the chosen installation type, the installation may take some time. During our tests (250Mbit down, 40Mbit up), the installation was usually finished within a 15-30 minute timeframe.
@@ -344,7 +376,7 @@ You can also login from your browser and access the Web UI: `https://<your.ip>:6
 
 <a name="placement"></a>
 # System Placement
-Make sure your system is reachable through the internet. Otherwise it will not capture any attacks, other than the ones from your  internal network! We recommend you put it in an unfiltered zone, where all TCP and UDP traffic is forwarded to T-Pot's network interface. However to avoid fingerprinting you can put T-Pot behind a firewall and forward all TCP / UDP traffic in the port range of 1-64000 to T-Pot while allowing access to ports > 64000 only from trusted IPs.
+Make sure your system is reachable through a network you suspect intruders in / from (i.e. the internet). Otherwise T-Pot will most likely not capture any attacks, other than the ones from your  internal network! We recommend you put it in an unfiltered zone, where all TCP and UDP traffic is forwarded to T-Pot's network interface. However to avoid fingerprinting you can put T-Pot behind a firewall and forward all TCP / UDP traffic in the port range of 1-64000 to T-Pot while allowing access to ports > 64000 only from trusted IPs.
 
 A list of all relevant ports is available as part of the [Technical Concept](#concept)
 <br>
@@ -355,21 +387,23 @@ In case you need external Admin UI access, forward TCP port 64294 to T-Pot, see
 In case you need external SSH access, forward TCP port 64295 to T-Pot, see below.
 In case you need external Web UI access, forward TCP port 64297 to T-Pot, see below.
 
-T-Pot requires outgoing git, http, https connections for updates (Ubuntu, Docker, GitHub, PyPi) and attack submission (ewsposter, hpfeeds). Ports and availability may vary based on your geographical location.
+T-Pot requires outgoing git, http, https connections for updates (Debian, Docker, GitHub, PyPi) and attack submission (ewsposter, hpfeeds). Ports and availability may vary based on your geographical location.
 
 <a name="updates"></a>
 # Updates
 For the ones of you who want to live on the bleeding edge of T-Pot development we introduced an update feature which will allow you to update all T-Pot relevant files to be up to date with the T-Pot master branch.
 **If you made any relevant changes to the T-Pot relevant config files make sure to create a backup first.**
-- The Update script will
- - **merciless** overwrite local changes to be in sync with the T-Pot master branch
- - upgrade the system to the latest kernel within Ubuntu 18.04.x LTS
- - upgrade the system to the latest packages available within Ubuntu 18.04.x LTS
- - update all resources to be en par with the T-Pot master branch
- - ensure all T-Pot relevant system files will be patched / copied into original T-Pot state
+
+The Update script will:
+ - **mercilessly** overwrite local changes to be in sync with the T-Pot master branch
+ - upgrade the system to the packages available in Debian (Sid)
+ - update all resources to be in-sync with the T-Pot master branch
+ - ensure all T-Pot relevant system files will be patched / copied into the original T-Pot state
+ - restore your custom ews.cfg and HPFEED settings from `/data/ews/conf`
 
 You simply run the update script:
 ```
+sudo su -
 cd /opt/tpot/
 ./update.sh -y
 ```
@@ -428,7 +462,7 @@ If new versions of the components involved appear, we will test them and build n
 
 <a name="submission"></a>
 ## Community Data Submission
-We provide T-Pot in order to make it accessible to all parties interested in honeypot deployment. By default, the captured data is submitted to a community backend. This community backend uses the data to feed [Sicherheitstacho](https://sicherheitstacho.eu.
+We provide T-Pot in order to make it accessible to all parties interested in honeypot deployment. By default, the captured data is submitted to a community backend. This community backend uses the data to feed [Sicherheitstacho](https://sicherheitstacho.eu).
 You may opt out of the submission by removing the `# Ewsposter service` from `/opt/tpot/etc/tpot.yml`:
 1. Stop T-Pot services: `systemctl stop tpot`
 2. Remove Ewsposter service: `vi /opt/tpot/etc/tpot.yml`
@@ -440,7 +474,7 @@ You may opt out of the submission by removing the `# Ewsposter service` from `/o
     restart: always
     networks:
      - ewsposter_local
-    image: "dtagdevsec/ewsposter:1810"
+    image: "dtagdevsec/ewsposter:1903"
     volumes:
      - /data:/data
      - /data/ews/conf/ews.ip:/opt/ewsposter/ews.ip
@@ -451,6 +485,20 @@ Data is submitted in a structured ews-format, a XML stucture. Hence, you can par
 
 We encourage you not to disable the data submission as it is the main purpose of the community approach - as you all know **sharing is caring** 😍
 
+<a name="hpfeeds-optin"></a>
+## Opt-In HPFEEDS Data Submission
+As an Opt-In it is now possible to also share T-Pot data with 3rd party HPFEEDS brokers, such as [SISSDEN](https://sissden.eu).  
+If you want to share your T-Pot data you simply have to register an account with a 3rd party broker with its own benefits towards the community. Once registered you will receive your credentials to share events with the broker. In T-Pot you simply run `hpfeeds_optin.sh` which will ask for your credentials, in case of SISSDEN this is just `Ident` and `Secret`, everything else is pre-configured.  
+It will automatically update `/opt/tpot/etc/tpot.yml` to deliver events to your desired broker.
+
+The script can accept a config file as an argument, e.g. `./hpfeeds_optin.sh --conf=hpfeeds.cfg`
+
+Your current config will also be stored in `/data/ews/conf/hpfeeds.cfg` where you can review or change it.  
+Be sure to apply any changes by running `./hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg`.  
+No worries: Your old config gets backed up in `/data/ews/conf/hpfeeds.cfg.old`
+
+Of course you can also rerun the `hpfeeds_optin.sh` script to change and apply your settings interactively.
+
 <a name="roadmap"></a>
 # Roadmap
 As with every development there is always room for improvements ...
@@ -479,32 +527,36 @@ We hope you understand that we cannot provide support on an individual basis. We
 <a name="licenses"></a>
 # Licenses
 The software that T-Pot is built on uses the following licenses.
-<br>GPLv2: [conpot)](https://github.com/mushorg/conpot/blob/master/LICENSE.txt), [dionaea](https://github.com/DinoTools/dionaea/blob/master/LICENSE), [honeytrap](https://github.com/armedpot/honeytrap/blob/master/LICENSE), [suricata](http://suricata-ids.org/about/open-source/)
-<br>GPLv3: [adbhoney](https://github.com/huuck/ADBHoney), [elasticpot](https://github.com/schmalle/ElasticPot), [ewsposter](https://github.com/dtag-dev-sec/ews/), [glastopf](https://github.com/glastopf/glastopf/blob/master/GPL), [rdpy](https://github.com/citronneur/rdpy/blob/master/LICENSE), [heralding](https://github.com/johnnykv/heralding/blob/master/LICENSE.txt), [snare](https://github.com/mushorg/snare/blob/master/LICENSE), [tanner](https://github.com/mushorg/snare/blob/master/LICENSE)
+<br>GPLv2: [conpot](https://github.com/mushorg/conpot/blob/master/LICENSE.txt), [dionaea](https://github.com/DinoTools/dionaea/blob/master/LICENSE), [honeypy](https://github.com/foospidy/HoneyPy/blob/master/LICENSE), [honeytrap](https://github.com/armedpot/honeytrap/blob/master/LICENSE), [suricata](http://suricata-ids.org/about/open-source/)
+<br>GPLv3: [adbhoney](https://github.com/huuck/ADBHoney), [elasticpot](https://github.com/schmalle/ElasticpotPY), [ewsposter](https://github.com/dtag-dev-sec/ews/), [fatt](https://github.com/0x4D31/fatt/blob/master/LICENSE), [rdpy](https://github.com/citronneur/rdpy/blob/master/LICENSE), [heralding](https://github.com/johnnykv/heralding/blob/master/LICENSE.txt), [snare](https://github.com/mushorg/snare/blob/master/LICENSE), [tanner](https://github.com/mushorg/snare/blob/master/LICENSE)
 <br>Apache 2 License: [cyberchef](https://github.com/gchq/CyberChef/blob/master/LICENSE), [elasticsearch](https://github.com/elasticsearch/elasticsearch/blob/master/LICENSE.txt), [logstash](https://github.com/elasticsearch/logstash/blob/master/LICENSE), [kibana](https://github.com/elasticsearch/kibana/blob/master/LICENSE.md), [docker](https://github.com/docker/docker/blob/master/LICENSE), [elasticsearch-head](https://github.com/mobz/elasticsearch-head/blob/master/LICENCE)
-<br>MIT license: [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot/blob/master/LICENSE), [ctop](https://github.com/bcicen/ctop/blob/master/LICENSE), [glutton](https://github.com/mushorg/glutton/blob/master/LICENSE)
-<br> Other: [cowrie](https://github.com/micheloosterhof/cowrie/blob/master/LICENSE.md), [mailoney](https://github.com/awhitehatter/mailoney), [Ubuntu licensing](http://www.ubuntu.com/about/about-ubuntu/licensing)
+<br>MIT license: [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot/blob/master/LICENSE), [glutton](https://github.com/mushorg/glutton/blob/master/LICENSE)
+<br> Other: [citrixhoneypot](https://github.com/MalwareTech/CitrixHoneypot#licencing-agreement-malwaretech-public-licence), [cowrie](https://github.com/micheloosterhof/cowrie/blob/master/LICENSE.md), [mailoney](https://github.com/awhitehatter/mailoney), [Debian licensing](https://www.debian.org/legal/licenses/)
 
 <a name="credits"></a>
 # Credits
-Without open source and the fruitful development community we are proud to be a part of, T-Pot would not have been possible! Our thanks are extended but not limited to the following people and organizations:
+Without open source and the fruitful development community (we are proud to be a part of), T-Pot would not have been possible! Our thanks are extended but not limited to the following people and organizations:
 
 ### The developers and development communities of
 
 * [adbhoney](https://github.com/huuck/ADBHoney/graphs/contributors)
+* [apt-fast](https://github.com/ilikenwf/apt-fast/graphs/contributors)
 * [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot/graphs/contributors)
+* [citrixhoneypot](https://github.com/MalwareTech/CitrixHoneypot/graphs/contributors)
 * [cockpit](https://github.com/cockpit-project/cockpit/graphs/contributors)
 * [conpot](https://github.com/mushorg/conpot/graphs/contributors)
 * [cowrie](https://github.com/micheloosterhof/cowrie/graphs/contributors)
+* [debian](http://www.debian.org/)
 * [dionaea](https://github.com/DinoTools/dionaea/graphs/contributors)
 * [docker](https://github.com/docker/docker/graphs/contributors)
-* [elasticpot](https://github.com/schmalle/ElasticPot/graphs/contributors)
+* [elasticpot](https://github.com/schmalle/ElasticpotPY/graphs/contributors)
 * [elasticsearch](https://github.com/elastic/elasticsearch/graphs/contributors)
 * [elasticsearch-head](https://github.com/mobz/elasticsearch-head/graphs/contributors)
 * [ewsposter](https://github.com/armedpot/ewsposter/graphs/contributors)
-* [glastopf](https://github.com/mushorg/glastopf/graphs/contributors)
+* [fatt](https://github.com/0x4D31/fatt/graphs/contributors)
 * [glutton](https://github.com/mushorg/glutton/graphs/contributors)
 * [heralding](https://github.com/johnnykv/heralding/graphs/contributors)
+* [honeypy](https://github.com/foospidy/HoneyPy/graphs/contributors)
 * [honeytrap](https://github.com/armedpot/honeytrap/graphs/contributors)
 * [kibana](https://github.com/elastic/kibana/graphs/contributors)
 * [logstash](https://github.com/elastic/logstash/graphs/contributors)
@@ -516,10 +568,9 @@ Without open source and the fruitful development community we are proud to be a
 * [snare](https://github.com/mushorg/snare/graphs/contributors)
 * [tanner](https://github.com/mushorg/tanner/graphs/contributors)
 * [suricata](https://github.com/inliniac/suricata/graphs/contributors)
-* [ubuntu](http://www.ubuntu.com/)
 
 ### The following companies and organizations
-* [canonical](http://www.canonical.com/)
+* [debian](https://www.debian.org/)
 * [docker](https://www.docker.com/)
 * [elastic.io](https://www.elastic.co/)
 * [honeynet project](https://www.honeynet.org/)
@@ -531,7 +582,12 @@ Without open source and the fruitful development community we are proud to be a
 # Stay tuned ...
 We will be releasing a new version of T-Pot about every 6-12 months.
 
+<a name="testimonial"></a>
+# Testimonial
+One of the greatest feedback we have gotten so far is by one of the Conpot developers:<br>
+***"[...] I highly recommend T-Pot which is ... it's not exactly a swiss army knife .. it's more like a swiss army soldier, equipped with a swiss army knife. Inside a tank. A swiss tank. [...]"***
+
 <a name="funfact"></a>
 # Fun Fact
 
-In an effort of saving the environment we are now brewing our own Mate Ice Tea and consumed 241 liters so far for the T-Pot 18.11 development 😇
+In an effort of saving the environment we are now brewing our own Mate Ice Tea and consumed 73 liters so far for the T-Pot 19.03 development 😇

bin/backup_es_folders.sh

@@ -1,4 +1,12 @@
 #!/bin/bash
+# Run as root only.
+myWHOAMI=$(whoami)
+if [ "$myWHOAMI" != "root" ]
+  then
+    echo "Need to run as root ..."
+    exit
+fi
+
 # Backup all ES relevant folders
 # Make sure ES is available
 myES="http://127.0.0.1:64298/"
@@ -16,7 +24,7 @@ fi
 myCOUNT=1
 myDATE=$(date +%Y%m%d%H%M)
 myELKPATH="/data/elk/data"
-myKIBANAINDEXNAME=$(curl -s -XGET ''$myES'_cat/indices/' | grep -w ".kibana_1" | awk '{ print $4 }')
+myKIBANAINDEXNAME=$(curl -s -XGET ''$myES'_cat/indices/.kibana' | awk '{ print $4 }')
 myKIBANAINDEXPATH=$myELKPATH/nodes/0/indices/$myKIBANAINDEXNAME
 
 # Let's ensure normal operation on exit or if interrupted ...

bin/change_ews_config.sh

@@ -0,0 +1,89 @@
+#!/bin/bash
+
+echo """
+
+##############################
+# T-POT DTAG Data Submission #
+# Contact:                   #
+# cert@telekom.de            # 
+##############################
+"""
+
+# Got root?
+myWHOAMI=$(whoami)
+if [ "$myWHOAMI" != "root" ]
+  then
+    echo "Need to run as root ..."
+    sudo ./$0
+    exit
+fi
+
+printf "[*] Enter your API UserID: "
+read apiUser
+printf "[*] Enter your API Token: "
+read apiToken
+printf "[*] If you have multiple T-Pots running, give them each a unique NUMBER, e.g. '2' for your second T-Pot installation. Enter unique number for THIS T-Pot: "
+read indexNumber
+if ! [[ "$indexNumber" =~ ^[0-9]+$ ]]
+    then
+        echo "Sorry integers only. You have to start over..."
+        exit 1
+fi
+apiURL="https://community.sicherheitstacho.eu/ews-0.1/alert/postSimpleMessage"
+printf "[*] Currently, your honeypot is configured to transmit data the default backend at 'https://community.sicherheitstacho.eu/ews-0.1/alert/postSimpleMessage'. Do you want to change this API endpoint? Only do this if you run your own PEBA backend instance? (N/y): "
+read replyAPI
+if [[ $replyAPI =~ ^[Yy]$ ]]
+then    
+    printf "[*] Enter your API endpoint URL and make sure it contains the full path, e.g. 'https://myDomain.local:9922/ews-0.1/alert/postSimpleMessage': "
+    read apiURL
+fi
+
+
+
+echo ""
+echo "[*] Recap! You defined: "
+echo "############################"
+echo "API User: " $apiUser
+echo "API Token: " $apiToken
+echo "API URL: " $apiURL
+echo "Unique numeric ID for your T-Pot Installation: "  $indexNumber
+echo "Specific honeypot-IDs will look like : <honeypotType>-"$apiUser"-"$indexNumber
+echo "############################"
+echo ""
+printf  "[*] Is the above correct (y/N)? "
+read reply
+if [[ ! $reply =~ ^[Yy]$ ]]
+then	
+	echo "OK, then run this again..."
+    exit 1
+fi
+echo ""
+echo "[+] Creating config file with API UserID '$apiUser' and API Token '$apiToken'."
+echo "[+] Fetching config file from github. Outgoing https requests must be enabled!"
+wget -q https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/docker/ews/dist/ews.cfg -O ews.cfg.dist 
+if [[ -f "ews.cfg.dist" ]]; then
+	echo "[+] Successfully downloaded ews.cfg from github."
+else 
+	echo "[+] Could not download ews.cfg from github."
+	exit 1
+fi 
+echo "[+] Patching ews.cfg API Credentials."
+sed 's/community-01-user/'$apiUser'/' ews.cfg.dist > ews.cfg
+sed -i 's/foth{a5maiCee8fineu7/'$apiToken'/' ews.cfg
+echo "[+] Patching ews.cfg API Url."
+apiURL=${apiURL////\\/};
+sed -i 's/https:\/\/community.sicherheitstacho.eu\/ews-0.1\/alert\/postSimpleMessage/'$apiURL'/' ews.cfg
+echo "[+] Patching ews.cfg honeypot IDs."
+sed -i 's/community-01/'$apiUser'-'$indexNumber'/' ews.cfg
+
+rm ews.cfg.dist
+
+echo "[+] Changing tpot.yml to include new ews.cfg."
+
+cp ews.cfg /data/ews/conf/ews.cfg
+cp /opt/tpot/etc/tpot.yml /opt/tpot/etc/tpot.yml.bak
+sed -i '/- \/data\/ews\/conf\/ews.ip:\/opt\/ewsposter\/ews.ip/a\ \ \   - \/data\/ews\/conf\/ews.cfg:\/opt\/ewsposter\/ews.cfg' /opt/tpot/etc/tpot.yml
+
+echo "[+] Restarting T-Pot."
+systemctl restart tpot
+echo "[+] Done."

bin/clean.sh

@@ -1,11 +1,13 @@
 #!/bin/bash
 # T-Pot Container Data Cleaner & Log Rotator
-
 # Set colors
 myRED=""
 myGREEN=""
 myWHITE=""
 
+# Set pigz
+myPIGZ=$(which pigz)
+
 # Set persistence
 myPERSISTENCE=$1
 
@@ -38,7 +40,7 @@ fuLOGROTATE () {
   local myTANNERFTGZ="/data/tanner/files.tgz"
 
 # Ensure correct permissions and ownerships for logrotate to run without issues
-chmod 760 /data/ -R
+chmod 770 /data/ -R
 chown tpot:tpot /data -R
 chmod 644 /data/nginx/conf -R
 chmod 644 /data/nginx/cert -R
@@ -47,17 +49,17 @@ chmod 644 /data/nginx/cert -R
 logrotate -f -s $mySTATUS $myCONF
 
 # Compressing some folders first and rotate them later
-if [ "$(fuEMPTY $myADBHONEYDL)" != "0" ]; then tar cvfz $myADBHONEYTGZ $myADBHONEYDL; fi
-if [ "$(fuEMPTY $myCOWRIETTYLOGS)" != "0" ]; then tar cvfz $myCOWRIETTYTGZ $myCOWRIETTYLOGS; fi
-if [ "$(fuEMPTY $myCOWRIEDL)" != "0" ]; then tar cvfz $myCOWRIEDLTGZ $myCOWRIEDL; fi
-if [ "$(fuEMPTY $myDIONAEABI)" != "0" ]; then tar cvfz $myDIONAEABITGZ $myDIONAEABI; fi
-if [ "$(fuEMPTY $myDIONAEABIN)" != "0" ]; then tar cvfz $myDIONAEABINTGZ $myDIONAEABIN; fi
-if [ "$(fuEMPTY $myHONEYTRAPATTACKS)" != "0" ]; then tar cvfz $myHONEYTRAPATTACKSTGZ $myHONEYTRAPATTACKS; fi
-if [ "$(fuEMPTY $myHONEYTRAPDL)" != "0" ]; then tar cvfz $myHONEYTRAPDLTGZ $myHONEYTRAPDL; fi
-if [ "$(fuEMPTY $myTANNERF)" != "0" ]; then tar cvfz $myTANNERFTGZ $myTANNERF; fi
+if [ "$(fuEMPTY $myADBHONEYDL)" != "0" ]; then tar -I $myPIGZ -cvf $myADBHONEYTGZ $myADBHONEYDL; fi
+if [ "$(fuEMPTY $myCOWRIETTYLOGS)" != "0" ]; then tar -I $myPIGZ -cvf $myCOWRIETTYTGZ $myCOWRIETTYLOGS; fi
+if [ "$(fuEMPTY $myCOWRIEDL)" != "0" ]; then tar -I $myPIGZ -cvf $myCOWRIEDLTGZ $myCOWRIEDL; fi
+if [ "$(fuEMPTY $myDIONAEABI)" != "0" ]; then tar -I $myPIGZ -cvf $myDIONAEABITGZ $myDIONAEABI; fi
+if [ "$(fuEMPTY $myDIONAEABIN)" != "0" ]; then tar -I $myPIGZ -cvf $myDIONAEABINTGZ $myDIONAEABIN; fi
+if [ "$(fuEMPTY $myHONEYTRAPATTACKS)" != "0" ]; then tar -I $myPIGZ -cvf $myHONEYTRAPATTACKSTGZ $myHONEYTRAPATTACKS; fi
+if [ "$(fuEMPTY $myHONEYTRAPDL)" != "0" ]; then tar -I $myPIGZ -cvf $myHONEYTRAPDLTGZ $myHONEYTRAPDL; fi
+if [ "$(fuEMPTY $myTANNERF)" != "0" ]; then tar -I $myPIGZ -cvf $myTANNERFTGZ $myTANNERF; fi
 
 # Ensure correct permissions and ownership for previously created archives
-chmod 760 $myADBHONEYTGZ $myCOWRIETTYTGZ $myCOWRIEDLTGZ $myDIONAEABITGZ $myDIONAEABINTGZ $myHONEYTRAPATTACKSTGZ $myHONEYTRAPDLTGZ $myTANNERFTGZ
+chmod 770 $myADBHONEYTGZ $myCOWRIETTYTGZ $myCOWRIEDLTGZ $myDIONAEABITGZ $myDIONAEABINTGZ $myHONEYTRAPATTACKSTGZ $myHONEYTRAPDLTGZ $myTANNERFTGZ
 chown tpot:tpot $myADBHONEYTGZ $myCOWRIETTYTGZ $myCOWRIEDLTGZ $myDIONAEABITGZ $myDIONAEABINTGZ $myHONEYTRAPATTACKSTGZ $myHONEYTRAPDLTGZ $myTANNERFTGZ
 
 # Need to remove subfolders since too many files cause rm to exit with errors
@@ -65,7 +67,7 @@ rm -rf $myADBHONEYDL $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $my
 
 # Recreate subfolders with correct permissions and ownership
 mkdir -p $myADBHONEYDL $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL $myTANNERF
-chmod 760 $myADBHONEYDL $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL $myTANNERF
+chmod 770 $myADBHONEYDL $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL $myTANNERF
 chown tpot:tpot $myADBHONEYDL $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL $myTANNERF
 
 # Run logrotate again to account for previously created archives - DO NOT FORCE HERE!
@@ -76,7 +78,7 @@ logrotate -s $mySTATUS $myCONF
 fuADBHONEY () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/adbhoney/*; fi
   mkdir -p /data/adbhoney/log/ /data/adbhoney/downloads/
-  chmod 760 /data/adbhoney/ -R
+  chmod 770 /data/adbhoney/ -R
   chown tpot:tpot /data/adbhoney/ -R
 }
 
@@ -84,15 +86,23 @@ fuADBHONEY () {
 fuCISCOASA () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/ciscoasa/*; fi
   mkdir -p /data/ciscoasa/log
-  chmod 760 /data/ciscoasa -R
+  chmod 770 /data/ciscoasa -R
   chown tpot:tpot /data/ciscoasa -R
 }
 
+# Let's create a function to clean up and prepare citrixhoneypot data
+fuCITRIXHONEYPOT () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/citrixhoneypot/*; fi
+  mkdir -p /data/citrixhoneypot/logs/
+  chmod 770 /data/citrixhoneypot/ -R
+  chown tpot:tpot /data/citrixhoneypot/ -R
+}
+
 # Let's create a function to clean up and prepare conpot data
 fuCONPOT () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/conpot/*; fi
   mkdir -p /data/conpot/log
-  chmod 760 /data/conpot -R
+  chmod 770 /data/conpot -R
   chown tpot:tpot /data/conpot -R
 }
 
@@ -100,7 +110,7 @@ fuCONPOT () {
 fuCOWRIE () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/cowrie/*; fi
   mkdir -p /data/cowrie/log/tty/ /data/cowrie/downloads/ /data/cowrie/keys/ /data/cowrie/misc/
-  chmod 760 /data/cowrie -R
+  chmod 770 /data/cowrie -R
   chown tpot:tpot /data/cowrie -R
 }
 
@@ -108,7 +118,7 @@ fuCOWRIE () {
 fuDIONAEA () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/dionaea/*; fi
   mkdir -p /data/dionaea/log /data/dionaea/bistreams /data/dionaea/binaries /data/dionaea/rtp /data/dionaea/roots/ftp /data/dionaea/roots/tftp /data/dionaea/roots/www /data/dionaea/roots/upnp
-  chmod 760 /data/dionaea -R
+  chmod 770 /data/dionaea -R
   chown tpot:tpot /data/dionaea -R
 }
 
@@ -116,7 +126,7 @@ fuDIONAEA () {
 fuELASTICPOT () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/elasticpot/*; fi
   mkdir -p /data/elasticpot/log
-  chmod 760 /data/elasticpot -R
+  chmod 770 /data/elasticpot -R
   chown tpot:tpot /data/elasticpot -R
 }
 
@@ -126,23 +136,23 @@ fuELK () {
   # ELK daemon log files will be removed
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/elk/log/*; fi
   mkdir -p /data/elk
-  chmod 760 /data/elk -R
+  chmod 770 /data/elk -R
   chown tpot:tpot /data/elk -R
 }
 
-# Let's create a function to clean up and prepare glastopf data
-fuGLASTOPF () {
-  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/glastopf/*; fi
-  mkdir -p /data/glastopf/db /data/glastopf/log
-  chmod 760 /data/glastopf -R
-  chown tpot:tpot /data/glastopf -R
+# Let's create a function to clean up and prepare fatt data
+fuFATT () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/fatt/*; fi
+  mkdir -p /data/fatt/log
+  chmod 770 -R /data/fatt
+  chown tpot:tpot -R /data/fatt
 }
 
 # Let's create a function to clean up and prepare glastopf data
 fuGLUTTON () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/glutton/*; fi
   mkdir -p /data/glutton/log
-  chmod 760 /data/glutton -R
+  chmod 770 /data/glutton -R
   chown tpot:tpot /data/glutton -R
 }
 
@@ -150,15 +160,23 @@ fuGLUTTON () {
 fuHERALDING () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/heralding/*; fi
   mkdir -p /data/heralding/log
-  chmod 760 /data/heralding -R
+  chmod 770 /data/heralding -R
   chown tpot:tpot /data/heralding -R
 }
 
+# Let's create a function to clean up and prepare honeypy data
+fuHONEYPY () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/honeypy/*; fi
+  mkdir -p /data/honeypy/log
+  chmod 770 /data/honeypy -R
+  chown tpot:tpot /data/honeypy -R
+}
+
 # Let's create a function to clean up and prepare honeytrap data
 fuHONEYTRAP () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/honeytrap/*; fi
   mkdir -p /data/honeytrap/log/ /data/honeytrap/attacks/ /data/honeytrap/downloads/
-  chmod 760 /data/honeytrap/ -R
+  chmod 770 /data/honeytrap/ -R
   chown tpot:tpot /data/honeytrap/ -R
 }
 
@@ -166,7 +184,7 @@ fuHONEYTRAP () {
 fuMAILONEY () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/mailoney/*; fi
   mkdir -p /data/mailoney/log/
-  chmod 760 /data/mailoney/ -R
+  chmod 770 /data/mailoney/ -R
   chown tpot:tpot /data/mailoney/ -R
 }
 
@@ -174,7 +192,7 @@ fuMAILONEY () {
 fuMEDPOT () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/medpot/*; fi
   mkdir -p /data/medpot/log/
-  chmod 760 /data/medpot/ -R
+  chmod 770 /data/medpot/ -R
   chown tpot:tpot /data/medpot/ -R
 }
 
@@ -190,7 +208,7 @@ fuNGINX () {
 fuRDPY () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/rdpy/*; fi
   mkdir -p /data/rdpy/log/
-  chmod 760 /data/rdpy/ -R
+  chmod 770 /data/rdpy/ -R
   chown tpot:tpot /data/rdpy/ -R
 }
 
@@ -198,7 +216,7 @@ fuRDPY () {
 fuSPIDERFOOT () {
   mkdir -p /data/spiderfoot
   touch /data/spiderfoot/spiderfoot.db
-  chmod 760 -R /data/spiderfoot
+  chmod 770 -R /data/spiderfoot
   chown tpot:tpot -R /data/spiderfoot
 }
 
@@ -206,7 +224,7 @@ fuSPIDERFOOT () {
 fuSURICATA () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/suricata/*; fi
   mkdir -p /data/suricata/log
-  chmod 760 -R /data/suricata
+  chmod 770 -R /data/suricata
   chown tpot:tpot -R /data/suricata
 }
 
@@ -214,7 +232,7 @@ fuSURICATA () {
 fuP0F () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/p0f/*; fi
   mkdir -p /data/p0f/log
-  chmod 760 -R /data/p0f
+  chmod 770 -R /data/p0f
   chown tpot:tpot -R /data/p0f
 }
 
@@ -222,7 +240,7 @@ fuP0F () {
 fuTANNER () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/tanner/*; fi
   mkdir -p /data/tanner/log /data/tanner/files
-  chmod 760 -R /data/tanner
+  chmod 770 -R /data/tanner
   chown tpot:tpot -R /data/tanner
 }
 
@@ -250,14 +268,16 @@ if [ "$myPERSISTENCE" = "on" ];
     echo "Cleaning up and preparing data folders."
     fuADBHONEY
     fuCISCOASA
+    fuCITRIXHONEYPOT
     fuCONPOT
     fuCOWRIE
     fuDIONAEA
     fuELASTICPOT
     fuELK
-    fuGLASTOPF
+    fuFATT
     fuGLUTTON
     fuHERALDING
+    fuHONEYPY
     fuHONEYTRAP
     fuMAILONEY
     fuMEDPOT

bin/dps.sh

@@ -1,4 +1,13 @@
-#/bin/bash
+#!/bin/bash
+
+# Run as root only.
+myWHOAMI=$(whoami)
+if [ "$myWHOAMI" != "root" ]
+  then
+    echo "Need to run as root ..."
+    exit
+fi
+
 # Show current status of T-Pot containers
 myPARAM="$1"
 myCONTAINERS="$(cat /opt/tpot/etc/tpot.yml | grep -v '#' | grep container_name | cut -d: -f2 | sort | tr -d " ")"
@@ -9,14 +18,13 @@ myWHITE=""
 myMAGENTA=""
 
 function fuGETSTATUS {
-grc docker ps -f status=running -f status=exited --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}" | grep -v "NAME" | sort
+grc --colour=on docker ps -f status=running -f status=exited --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}" | grep -v "NAME" | sort
 }
 
 function fuGETSYS {
 printf "========| System |========\n"
 printf "%+10s %-20s\n" "Date: " "$(date)"
 printf "%+10s %-20s\n" "Uptime: " "$(uptime | cut -b 2-)"
-printf "%+10s %-20s\n" "CPU temp: " "$(sensors | grep 'Physical' | awk '{ print $4" " }' | tr -d [:cntrl:])"
 echo
 }
 

bin/dump_es.sh

@@ -2,10 +2,10 @@
 # Dump all ES data
 # Make sure ES is available
 myES="http://127.0.0.1:64298/"
-myESSTATUS=$(curl -s -XGET ''$myES'_cluster/health' | jq '.' | grep -c green)
+myESSTATUS=$(curl -s -XGET ''$myES'_cluster/health' | jq '.' | grep -c "green\|yellow")
 if ! [ "$myESSTATUS" = "1" ]
   then
-    echo "### Elasticsearch is not available, try starting via 'systemctl start elk'."
+    echo "### Elasticsearch is not available, try starting via 'systemctl start tpot'."
     exit
   else
     echo "### Elasticsearch is available, now continuing."
@@ -20,12 +20,12 @@ trap fuCLEANUP EXIT
 
 # Set vars
 myDATE=$(date +%Y%m%d%H%M)
-myINDICES=$(curl -s -XGET ''$myES'_cat/indices/' | awk '{ print $3 }' | sort | grep -v 1970)
-myES="http://127.0.0.1:64298/"
+myINDICES=$(curl -s -XGET ''$myES'_cat/indices/logstash-*' | awk '{ print $3 }' | sort | grep -v 1970)
+myINDICES+=" .kibana"
 myCOL1=""
 myCOL0=""
 
-# Dumping all ES data
+# Dumping Kibana and Logstash data
 echo $myCOL1"### The following indices will be dumped: "$myCOL0
 echo $myINDICES
 echo

bin/hpfeeds_optin.sh

@@ -0,0 +1,153 @@
+#!/bin/bash
+
+# Run as root only.
+myWHOAMI=$(whoami)
+if [ "$myWHOAMI" != "root" ]
+  then
+    echo "Need to run as root ..."
+    exit
+fi
+
+myTPOTYMLFILE="/opt/tpot/etc/tpot.yml"
+
+function fuSISSDEN () {
+echo
+echo "You chose SISSDEN, you just need to provide ident and secret"
+echo
+myENABLE="true"
+myHOST="hpfeeds.sissden.eu"
+myPORT="10000"
+myCHANNEL="t-pot.events"
+myCERT="/opt/ewsposter/sissden.pem"
+read -p "Ident: " myIDENT
+read -p "Secret: " mySECRET
+myFORMAT="json"
+}
+
+function fuGENERIC () {
+echo
+echo "You chose generic, please provide all the details of the broker"
+echo
+myENABLE="true"
+read -p "Host URL: " myHOST
+read -p "Port: " myPORT
+read -p "Channel: " myCHANNEL
+echo "For generic providers set this to 'false'"
+echo "If you received a CA certficate mount it into the ewsposter container by modifying $myTPOTYMLFILE"
+read -p "TLS - 'false' or path to CA in container: " myCERT
+read -p "Ident: " myIDENT
+read -p "Secret: " mySECRET
+read -p "Format ews (xml) or json: " myFORMAT
+}
+
+function fuOPTOUT () {
+echo
+while [ 1 != 2 ]
+  do
+    read -s -n 1 -p "You chose to opt out (y/n)? " mySELECT
+      echo $mySELECT
+      case "$mySELECT" in
+        [y,Y])
+          echo "Opt out."
+          break
+          ;;
+        [n,N])
+          echo "Aborted."
+          exit
+          ;;
+      esac
+done
+myENABLE="false"
+myHOST="host"
+myPORT="port"
+myCHANNEL="channels"
+myCERT="false"
+myIDENT="user"
+mySECRET="secret"
+myFORMAT="json"
+}
+
+function fuWRITETOFILE () {
+if [ -f '/data/ews/conf/hpfeeds.cfg' ]; then
+  echo "Creating backup of current config in /data/ews/conf/hpfeeds.cfg.old"
+  mv /data/ews/conf/hpfeeds.cfg /data/ews/conf/hpfeeds.cfg.old
+fi
+echo "Storing new config in /data/ews/conf/hpfeeds.cfg"
+cat >> /data/ews/conf/hpfeeds.cfg <<EOF
+myENABLE=$myENABLE
+myHOST=$myHOST
+myPORT=$myPORT
+myCHANNEL=$myCHANNEL
+myCERT=$myCERT
+myIDENT=$myIDENT
+mySECRET=$mySECRET
+myFORMAT=$myFORMAT
+EOF
+}
+
+function fuAPPLY () {
+echo "Now stopping T-Pot ..."
+systemctl stop tpot
+echo "Applying your settings to tpot.yml ... "
+sed --follow-symlinks -i "s/EWS_HPFEEDS_ENABLE.*/EWS_HPFEEDS_ENABLE=${myENABLE}/g" "$myTPOTYMLFILE"
+sed --follow-symlinks -i "s/EWS_HPFEEDS_HOST.*/EWS_HPFEEDS_HOST=${myHOST}/g" "$myTPOTYMLFILE"
+sed --follow-symlinks -i "s/EWS_HPFEEDS_PORT.*/EWS_HPFEEDS_PORT=${myPORT}/g" "$myTPOTYMLFILE"
+sed --follow-symlinks -i "s/EWS_HPFEEDS_CHANNELS.*/EWS_HPFEEDS_CHANNELS=${myCHANNEL}/g" "$myTPOTYMLFILE"
+sed --follow-symlinks -i "s#EWS_HPFEEDS_TLSCERT.*#EWS_HPFEEDS_TLSCERT=${myCERT}#g" "$myTPOTYMLFILE"
+sed --follow-symlinks -i "s/EWS_HPFEEDS_IDENT.*/EWS_HPFEEDS_IDENT=${myIDENT}/g" "$myTPOTYMLFILE"
+sed --follow-symlinks -i "s/EWS_HPFEEDS_SECRET.*/EWS_HPFEEDS_SECRET=${mySECRET}/g" "$myTPOTYMLFILE"
+sed --follow-symlinks -i "s/EWS_HPFEEDS_FORMAT.*/EWS_HPFEEDS_FORMAT=${myFORMAT}/g" "$myTPOTYMLFILE"
+echo "Now starting T-Pot ..."
+systemctl start tpot
+echo "You can always change or review your settings in /data/ews/conf/hpfeeds.cfg and apply changes by"
+echo "running \"./hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg\""
+echo "Done."
+}
+
+# Check for cmdline argument and parse config file
+filename=$(echo $@ | cut -d= -f2)
+if [ $# == 1 ] && echo $@ | grep '\-\-conf=' > /dev/null && [ ! -z $filename ] && [ -f $filename ]
+  then
+    source $filename
+else
+
+# Proceed with interactive setup when no config file is found
+echo "HPFEEDS Delivery Opt-In for T-Pot"
+echo "---------------------------------"
+echo "By running this script you agree to share your data with a 3rd party and agree to their corresponding sharing terms."
+echo
+echo
+echo "Please choose your broker"
+echo "---------------------------"
+echo "[1] - SISSDEN"
+echo "[2] - Generic (enter details manually)"
+echo "[0] - Opt out of HPFEEDS"
+echo "[q] - Do not agree end exit"
+echo
+while [ 1 != 2 ]
+  do
+    read -s -n 1 -p "Your choice: " mySELECT
+      echo $mySELECT
+      case "$mySELECT" in
+        [1])
+	  fuSISSDEN
+          break
+          ;;
+        [2])
+	  fuGENERIC
+          break
+          ;;
+        [0])
+	  fuOPTOUT
+          break
+          ;;
+        [q,Q])
+	  echo "Aborted."
+          exit
+          ;;
+      esac
+done
+
+fi
+fuWRITETOFILE
+fuAPPLY

bin/hptest.sh

@@ -0,0 +1,118 @@
+#!/bin/bash
+
+myHOST="$1"
+myPACKAGES="netcat nmap"
+myMEDPOTPACKET="
+MSH|^~\&|ADT1|MCM|LABADT|MCM|198808181126|SECURITY|ADT^A01|MSG00001-|P|2.6
+EVN|A01|198808181123
+PID|||PATID1234^5^M11^^AN||JONES^WILLIAM^A^III||19610615|M||2106-3|677 DELAWARE AVENUE^^EVERETT^MA^02149|GL|(919)379-1212|(919)271-3434~(919)277-3114||S||PATID12345001^2^M10^^ACSN|123456789|9-87654^NC
+NK1|1|JONES^BARBARA^K|SPO|||||20011105
+NK1|1|JONES^MICHAEL^A|FTH
+PV1|1|I|2000^2012^01||||004777^LEBAUER^SIDNEY^J.|||SUR||-||ADM|A0
+AL1|1||^PENICILLIN||CODE16~CODE17~CODE18
+AL1|2||^CAT DANDER||CODE257
+DG1|001|I9|1550|MAL NEO LIVER, PRIMARY|19880501103005|F
+PR1|2234|M11|111^CODE151|COMMON PROCEDURES|198809081123
+ROL|45^RECORDER^ROLE MASTER LIST|AD|RO|KATE^SMITH^ELLEN|199505011201
+GT1|1122|1519|BILL^GATES^A
+IN1|001|A357|1234|BCMD|||||132987
+IN2|ID1551001|SSN12345678
+ROL|45^RECORDER^ROLE MASTER LIST|AD|RO|KATE^ELLEN|199505011201"
+
+function fuGOTROOT {
+myWHOAMI=$(whoami)
+if [ "$myWHOAMI" != "root" ]
+  then
+    echo "Need to run as root ..."
+    exit
+fi
+}
+
+function fuCHECKDEPS {
+myINST=""
+for myDEPS in $myPACKAGES;
+do
+  myOK=$(dpkg -s $myDEPS | grep ok | awk '{ print $3 }');
+  if [ "$myOK" != "ok" ]
+    then
+      myINST=$(echo $myINST $myDEPS)
+  fi
+done
+if [ "$myINST" != "" ]
+  then
+    apt-get update -y
+    for myDEPS in $myINST;
+    do
+      apt-get install $myDEPS -y
+    done
+fi
+}
+
+function fuCHECKFORARGS {
+if [ "$myHOST" != "" ];
+  then
+    echo "All arguments met. Continuing."
+  else
+    echo "Usage: hp_test.sh <[host or ip]>"
+    exit
+fi
+}
+
+function fuGETPORTS {
+myDOCKERCOMPOSEPORTS=$(cat $myDOCKERCOMPOSEYML | yq -r '.services[].ports' | grep ':' | sed -e s/127.0.0.1// | tr -d '", ' | sed -e s/^:// | cut -f1 -d ':' | grep -v "6429\|6430" | sort -gu)
+myPORTS=$(for i in $myDOCKERCOMPOSEPORTS; do echo "$i"; done)
+echo "Found these ports enabled:"
+echo "$myPORTS"
+exit
+}
+
+function fuSCAN {
+local myTIMEOUT="$1"
+local mySCANPORT="$2"
+local mySCANIP="$3"
+local mySCANOPTS="$4"
+
+timeout --foreground ${myTIMEOUT} nmap ${mySCANOPTS} -T4 -v -p ${mySCANPORT} ${mySCANIP} &
+}
+
+# Main
+fuGOTROOT
+fuCHECKDEPS
+fuCHECKFORARGS
+
+echo "Starting scans ..."
+echo "$myMEDPOTPACKET" | nc "$myHOST" 2575 &
+curl -XGET "http://$myHOST:9200/logstash-*/_search" &
+echo "
I20100" | timeout --foreground 3 nc "$myHOST" 10001 &
+fuSCAN "180" "7,8,102,135,161,1025,1080,5000,9200" "$myHOST" "-sC -sS -sU -sV"
+fuSCAN "180" "2048,4096,5432" "$myHOST" "-sC -sS -sU -sV --version-light"
+fuSCAN "120" "20,21" "$myHOST" "--script=ftp* -sC -sS -sV"
+fuSCAN "120" "22" "$myHOST" "--script=ssh2-enum-algos,ssh-auth-methods,ssh-hostkey,ssh-publickey-acceptance,sshv1 -sC -sS -sV"
+fuSCAN "30" "22" "$myHOST" "--script=ssh-brute"
+fuSCAN "120" "23,2323,2324" "$myHOST" "--script=telnet-encryption,telnet-ntlm-info -sC -sS -sV --version-light"
+fuSCAN "120" "25" "$myHOST" "--script=smtp* -sC -sS -sV"
+fuSCAN "180" "42" "$myHOST" "-sC -sS -sV"
+fuSCAN "120" "69" "$myHOST" "--script=tftp-enum -sU"
+fuSCAN "120" "80,81,8080,8443" "$myHOST" "-sC -sS -sV"
+fuSCAN "120" "110,995" "$myHOST" "--script=pop3-capabilities,pop3-ntlm-info -sC -sS -sV --version-light"
+fuSCAN "30" "110,995" "$myHOST" "--script=pop3-brute -sS"
+fuSCAN "120" "143,993" "$myHOST" "--script=imap-capabilities,imap-ntlm-info -sC -sS -sV --version-light"
+fuSCAN "30" "143,993" "$myHOST" "--script=imap-brute -sS"
+fuSCAN "240" "445" "$myHOST" "--script=smb-vuln* -sS -sU"
+fuSCAN "120" "502" "$myHOST" "--script=modbus-discover -sS -sU"
+fuSCAN "120" "623" "$myHOST" "--script=ipmi-cipher-zero,ipmi-version,supermicro-ipmi -sS -sU"
+fuSCAN "30" "623" "$myHOST" "--script=ipmi-brute -sS -sU"
+fuSCAN "120" "1433" "$myHOST" "--script=ms-sql* -sS"
+fuSCAN "120" "1723" "$myHOST" "--script=pptp-version -sS"
+fuSCAN "120" "1883" "$myHOST" "--script=mqtt-subscribe -sS"
+fuSCAN "120" "2404" "$myHOST" "--script=iec-identify -sS"
+fuSCAN "120" "3306" "$myHOST" "--script=mysql-vuln* -sC -sS -sV"
+fuSCAN "120" "3389" "$myHOST" "--script=rdp* -sC -sS -sV"
+fuSCAN "120" "5000" "$myHOST" "--script=*upnp* -sS -sU"
+fuSCAN "120" "5060,5061" "$myHOST" "--script=sip-call-spoof,sip-enum-users,sip-methods -sS -sU"
+fuSCAN "120" "5900" "$myHOST" "--script=vnc-info,vnc-title,realvnc-auth-bypass -sS"
+fuSCAN "120" "27017" "$myHOST" "--script=mongo* -sS"
+fuSCAN "120" "47808" "$myHOST" "--script=bacnet* -sS"
+wait
+reset
+echo "Done."

bin/restore_es.sh

@@ -2,10 +2,10 @@
 # Restore folder based ES backup
 # Make sure ES is available
 myES="http://127.0.0.1:64298/"
-myESSTATUS=$(curl -s -XGET ''$myES'_cluster/health' | jq '.' | grep -c green)
+myESSTATUS=$(curl -s -XGET ''$myES'_cluster/health' | jq '.' | grep -c "green\|yellow")
 if ! [ "$myESSTATUS" = "1" ]
   then
-    echo "### Elasticsearch is not available, try starting via 'systemctl start elk'."
+    echo "### Elasticsearch is not available, try starting via 'systemctl start tpot'."
     exit
   else
     echo "### Elasticsearch is available, now continuing."
@@ -41,13 +41,46 @@ echo $myCOL1"### Now unpacking tar archive: "$myDUMP $myCOL0
 tar xvf $myDUMP
 
 # Build indices list
-myINDICES=$(ls tmp/logstash*.gz | cut -c 5- | rev | cut -c 4- | rev)
+myINDICES="$(ls tmp/logstash*.gz | cut -c 5- | rev | cut -c 4- | rev)"
 myINDICES+=" .kibana"
 echo $myCOL1"### The following indices will be restored: "$myCOL0
 echo $myINDICES
 echo
 
+# Force single seat template for everything
+echo -n $myCOL1"### Forcing single seat template: "$myCOL0
+curl -s XPUT ''$myES'_template/.*' -H 'Content-Type: application/json' -d'
+{ "index_patterns": ".*",
+  "order": 1,
+  "settings": 
+    { 
+      "number_of_shards": 1,
+      "number_of_replicas": 0 
+    }
+}'
+echo
+
+# Set logstash template
+echo -n $myCOL1"### Setting up logstash template: "$myCOL0
+curl -s XPUT ''$myES'_template/logstash' -H 'Content-Type: application/json' -d'
+{
+  "index_patterns": "logstash-*",
+    "settings" : {
+      "index" : { 
+        "number_of_shards": 1,
+        "number_of_replicas": 0,
+          "mapping" : { 
+            "total_fields" : {
+              "limit" : "2000"
+            } 
+          }  
+      }
+    }
+}'
+echo
+
 # Restore indices
+curl -s -X DELETE ''$myES'.kibana*' > /dev/null
 for i in $myINDICES;
   do
     # Delete index if it already exists

bin/rules.sh

@@ -23,10 +23,10 @@ function fuNFQCHECK {
 myNFQCHECK=$(grep -e '^\s*honeytrap:\|^\s*glutton:' $myDOCKERCOMPOSEYML | tr -d ': ' | uniq)
 if [ "$myNFQCHECK" == "" ];
   then
-    echo "No NFQ related honeypot detected, no iptables rules needed. Exiting."
+    echo "No NFQ related honeypot detected, no iptables-legacy rules needed. Exiting."
     exit
   else
-    echo "Detected $myNFQCHECK as NFQ based honeypot, iptables rules needed. Continuing."
+    echo "Detected $myNFQCHECK as NFQ based honeypot, iptables-legacy rules needed. Continuing."
 fi
 }
 
@@ -41,54 +41,54 @@ echo "$myRULESPORTS"
 }
 
 function fuSETRULES {
-### Setting up iptables rules for honeytrap
+### Setting up iptables-legacy rules for honeytrap
 if [ "$myNFQCHECK" == "honeytrap" ];
   then
-    /sbin/iptables -w -A INPUT -s 127.0.0.1 -j ACCEPT
-    /sbin/iptables -w -A INPUT -d 127.0.0.1 -j ACCEPT
+    /usr/sbin/iptables-legacy -w -A INPUT -s 127.0.0.1 -j ACCEPT
+    /usr/sbin/iptables-legacy -w -A INPUT -d 127.0.0.1 -j ACCEPT
 
     for myPORT in $myRULESPORTS; do
-      /sbin/iptables -w -A INPUT -p tcp --dport $myPORT -j ACCEPT
+      /usr/sbin/iptables-legacy -w -A INPUT -p tcp --dport $myPORT -j ACCEPT
     done
 
-    /sbin/iptables -w -A INPUT -p tcp --syn -m state --state NEW -j NFQUEUE
+    /usr/sbin/iptables-legacy -w -A INPUT -p tcp --syn -m state --state NEW -j NFQUEUE
 fi
 
-### Setting up iptables rules for glutton
+### Setting up iptables-legacy rules for glutton
 if [ "$myNFQCHECK" == "glutton" ];
   then
-    /sbin/iptables -w -t raw -A PREROUTING -s 127.0.0.1 -j ACCEPT
-    /sbin/iptables -w -t raw -A PREROUTING -d 127.0.0.1 -j ACCEPT
+    /usr/sbin/iptables-legacy -w -t raw -A PREROUTING -s 127.0.0.1 -j ACCEPT
+    /usr/sbin/iptables-legacy -w -t raw -A PREROUTING -d 127.0.0.1 -j ACCEPT
 
     for myPORT in $myRULESPORTS; do
-      /sbin/iptables -w -t raw -A PREROUTING -p tcp --dport $myPORT -j ACCEPT
+      /usr/sbin/iptables-legacy -w -t raw -A PREROUTING -p tcp --dport $myPORT -j ACCEPT
     done
     # No need for NFQ forwarding, such rules are set up by glutton
 fi
 }
 
 function fuUNSETRULES {
-### Removing  iptables rules for honeytrap
+### Removing  iptables-legacy rules for honeytrap
 if [ "$myNFQCHECK" == "honeytrap" ];
   then
-    /sbin/iptables -w -D INPUT -s 127.0.0.1 -j ACCEPT
-    /sbin/iptables -w -D INPUT -d 127.0.0.1 -j ACCEPT
+    /usr/sbin/iptables-legacy -w -D INPUT -s 127.0.0.1 -j ACCEPT
+    /usr/sbin/iptables-legacy -w -D INPUT -d 127.0.0.1 -j ACCEPT
 
     for myPORT in $myRULESPORTS; do
-      /sbin/iptables -w -D INPUT -p tcp --dport $myPORT -j ACCEPT
+      /usr/sbin/iptables-legacy -w -D INPUT -p tcp --dport $myPORT -j ACCEPT
     done
 
-    /sbin/iptables -w -D INPUT -p tcp --syn -m state --state NEW -j NFQUEUE
+    /usr/sbin/iptables-legacy -w -D INPUT -p tcp --syn -m state --state NEW -j NFQUEUE
 fi
 
-### Removing iptables rules for glutton
+### Removing iptables-legacy rules for glutton
 if [ "$myNFQCHECK" == "glutton" ];
   then
-    /sbin/iptables -w -t raw -D PREROUTING -s 127.0.0.1 -j ACCEPT
-    /sbin/iptables -w -t raw -D PREROUTING -d 127.0.0.1 -j ACCEPT
+    /usr/sbin/iptables-legacy -w -t raw -D PREROUTING -s 127.0.0.1 -j ACCEPT
+    /usr/sbin/iptables-legacy -w -t raw -D PREROUTING -d 127.0.0.1 -j ACCEPT
 
     for myPORT in $myRULESPORTS; do
-      /sbin/iptables -w -t raw -D PREROUTING -p tcp --dport $myPORT -j ACCEPT
+      /usr/sbin/iptables-legacy -w -t raw -D PREROUTING -p tcp --dport $myPORT -j ACCEPT
     done
     # No need for removing NFQ forwarding, such rules are removed by glutton
 fi

bin/tped.sh

@@ -1,5 +1,13 @@
 #!/bin/bash
 
+# Run as root only.
+myWHOAMI=$(whoami)
+if [ "$myWHOAMI" != "root" ]
+  then
+    echo "Need to run as root ..."
+    exit
+fi
+
 # set backtitle, get filename
 myBACKTITLE="T-Pot Edition Selection Tool"
 myYMLS=$(cd /opt/tpot/etc/compose/ && ls -1 *.yml)
@@ -21,7 +29,7 @@ for i in $myYMLS;
   do
     myITEMS+="$i $(echo $i | cut -d "." -f1 | tr [:lower:] [:upper:]) " 
 done
-myEDITION=$(dialog --backtitle "$myBACKTITLE" --menu "Select T-Pot Edition" 13 50 6 $myITEMS 3>&1 1>&2 2>&3 3>&-)
+myEDITION=$(dialog --backtitle "$myBACKTITLE" --menu "Select T-Pot Edition" 12 50 5 $myITEMS 3>&1 1>&2 2>&3 3>&-)
 if [ "$myEDITION" == "" ];
   then
     echo "Have a nice day!"

bin/unlock_es.sh

@@ -0,0 +1,19 @@
+#/bin/bash
+# Unlock all ES indices for read / write mode
+# Useful in cases where ES locked all indices after disk quota has been reached
+# Make sure ES is available
+myES="http://127.0.0.1:64298/"
+myESSTATUS=$(curl -s -XGET ''$myES'_cluster/health' | jq '.' | grep -c "green\|yellow")
+if ! [ "$myESSTATUS" = "1" ]
+  then
+    echo "### Elasticsearch is not available, try starting via 'systemctl start tpot'."
+    exit
+  else
+    echo "### Elasticsearch is available, now continuing."
+    echo
+fi
+
+echo "### Trying to unlock all ES indices for read / write operation: "
+curl -XPUT -H "Content-Type: application/json" ''$myES'_all/_settings' -d '{"index.blocks.read_only_allow_delete": null}'
+echo
+

bin/updateip.sh

@@ -9,10 +9,18 @@ if [ "$myEXTIP" = "" ];
     myEXTIP=$myLOCALIP
 fi
 mySSHUSER=$(cat /etc/passwd | grep 1000 | cut -d ':' -f1)
-sed -i "s#IP:.*#IP: $myLOCALIP ($myEXTIP)#" /etc/issue
-sed -i "s#SSH:.*#SSH: ssh -l tsec -p 64295 $myLOCALIP#" /etc/issue
-sed -i "s#WEB:.*#WEB: https://$myLOCALIP:64297#" /etc/issue
-sed -i "s#ADMIN:.*#ADMIN: https://$myLOCALIP:64294#" /etc/issue
+echo "" > /etc/issue
+toilet -f ivrit -F metal --filter border:metal "T-Pot   19.03" | sed 's/\\/\\\\/g' >> /etc/issue
+echo >> /etc/issue
+echo ",---- [ \n ] [ \d ] [ \t ]" >> /etc/issue
+echo "|" >> /etc/issue
+echo "| IP: $myLOCALIP ($myEXTIP)" >> /etc/issue
+echo "| SSH: ssh -l tsec -p 64295 $myLOCALIP" >> /etc/issue 
+echo "| WEB: https://$myLOCALIP:64297" >> /etc/issue
+echo "| ADMIN: https://$myLOCALIP:64294" >> /etc/issue
+echo "|" >> /etc/issue
+echo "\`----" >> /etc/issue
+echo >> /etc/issue
 tee /data/ews/conf/ews.ip << EOF
 [MAIN]
 ip = $myEXTIP
@@ -23,4 +31,4 @@ MY_INTIP=$myLOCALIP
 MY_HOSTNAME=$HOSTNAME
 EOF
 chown tpot:tpot /data/ews/conf/ews.ip
-chmod 760 /data/ews/conf/ews.ip
+chmod 770 /data/ews/conf/ews.ip

cloud/ansible/.gitignore

@@ -0,0 +1,2 @@
+# Ansible
+*.retry

cloud/ansible/README.md

@@ -0,0 +1,235 @@
+# T-Pot Ansible
+
+Here you can find a ready-to-use solution for your automated T-Pot deployment using [Ansible](https://www.ansible.com/).  
+It consists of an Ansible Playbook with multiple roles, which is reusable for all [OpenStack](https://www.openstack.org/) based clouds (e.g. Open Telekom Cloud, Orange Cloud, Telefonica Open Cloud, OVH) out of the box.  
+Apart from that you can easily adapt the deploy role to use other [cloud providers](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html) (e.g. AWS, Azure, Digital Ocean, Google).
+
+The Playbook first creates all resources (security group, network, subnet, router), deploys a new server and then installs and configures T-Pot.
+
+This example showcases the deployment on our own OpenStack based Public Cloud Offering [Open Telekom Cloud](https://open-telekom-cloud.com/en).
+
+# Table of contents
+- [Preparation of Ansible Master](#ansible-master)
+  - [Ansible Installation](#ansible)
+  - [Agent Forwarding](#agent-forwarding)
+- [Preparations in Open Telekom Cloud Console](#preparation)
+  - [Create new project](#project)
+  - [Create API user](#api-user)
+  - [Import Key Pair](#key-pair)
+- [Clone Git Repository](#clone-git)
+- [Settings and recommended values](#settings)
+  - [OpenStack authentication variables](#os-auth)
+  - [Ansible remote user](#remote-user)
+  - [Instance settings](#instance-settings)
+  - [User password](#user-password)
+  - [Configure `tpot.conf.dist`](#tpot-conf)
+  - [Optional: Custom `ews.cfg`](#ews-cfg)
+  - [Optional: Custom HPFEEDS](#hpfeeds)
+- [Deploying a T-Pot](#deploy)
+- [Further documentation](#documentation)
+
+<a name="ansible-master"></a>
+# Preparation of Ansible Master
+You can either run the Ansible Playbook locally on your Linux or macOS machine or you can use an ECS (Elastic Cloud Server) on Open Telekom Cloud, which I did.  
+I used Ubuntu 18.04 for my Ansible Master Server, but other OSes are fine too.  
+Ansible works over the SSH Port, so you don't have to add any special rules to your Security Group.
+
+<a name="ansible"></a>
+## Ansible Installation
+Example for Ubuntu 18.04:  
+
+At first we update the system:  
+`sudo apt update`  
+`sudo apt dist-upgrade`
+
+Then we need to add the repository and install Ansible:  
+`sudo apt-add-repository --yes --update ppa:ansible/ansible`  
+`sudo apt install ansible`
+
+For other OSes and Distros have a look at the official [Ansible Documentation](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html).
+
+<a name="agent-forwarding"></a>
+## Agent Forwarding
+If you run the Ansible Playbook remotely on your Ansible Master Server, Agent Forwarding must be enabled in order to let Ansible connect to newly created machines.  
+- On Linux or macOS:  
+  - Create or edit `~/.ssh/config`
+    ```
+    Host ANSIBLE_MASTER_IP
+    ForwardAgent yes
+    ```
+- On Windows using Putty:  
+![Putty Agent Forwarding](doc/putty_agent_forwarding.png)
+
+<a name="preparation"></a>
+# Preparations in Open Telekom Cloud Console
+(You can skip this if you have already set up a project and an API account with key pair)  
+(Just make sure you know the naming for everything, as you need to configure the Ansible variables.)
+
+Before we can start deploying, we have to prepare the Open Telekom Cloud tenant.  
+For that, go to the [Web Console](https://auth.otc.t-systems.com/authui/login) and log in with an admin user.
+
+<a name="project"></a>
+## Create new project
+I strongly advise you to create a separate project for the T-Pots in your tenant.  
+In my case I named it `tpot`.
+
+![Create new project](doc/otc_1_project.gif)
+
+<a name="api-user"></a>
+## Create API user
+The next step is to create a new user account, which is restricted to the project.  
+This ensures that the API access is limited to that project.
+
+![Create API user](doc/otc_2_user.gif)
+
+<a name="key-pair"></a>
+## Import Key Pair
+:warning: Now log in with the newly created API user account and select your project.
+
+![Login as API user](doc/otc_3_login.gif)
+
+Import your SSH public key.
+
+![Import SSH Public Key](doc/otc_4_import_key.gif)
+
+
+<a name="clone-git"></a>
+# Clone Git Repository
+Clone the `tpotce` repository to your Ansible Master:  
+`git clone https://github.com/dtag-dev-sec/tpotce.git`  
+All Ansible related files are located in the [`cloud/ansible/openstack`](../../cloud/ansible/openstack) folder.
+
+<a name="settings"></a>
+# Settings and recommended values
+You can configure all aspects of your Elastic Cloud Server and T-Pot before using the Playbook.  
+The settings are located in the following Ansible vars files:
+
+<a name="os-auth"></a>
+## OpenStack authentication variables
+Located at [`openstack/roles/deploy/vars/os_auth.yaml`](openstack/roles/deploy/vars/os_auth.yaml).  
+Enter your Open Telekom Cloud API user credentials here (username, password, project name, user domain name):  
+```
+auth_url: https://iam.eu-de.otc.t-systems.com/v3
+username: your_api_user
+password: your_password
+project_name: eu-de_your_project
+os_user_domain_name: OTC-EU-DE-000000000010000XXXXX
+```
+You can also perform different authentication methods like sourcing your `.ostackrc` file or using the OpenStack `clouds.yaml` file.  
+For more information have a look in the [os_server](https://docs.ansible.com/ansible/latest/modules/os_server_module.html) Ansible module documentation.
+
+<a name="remote-user"></a>
+## Ansible remote user
+You may have to adjust the `remote_user` in the Ansible Playbook under [`openstack/deploy_tpot.yaml`](openstack/deploy_tpot.yaml) depending on your Debian base image (e.g. on Open Telekom Cloud the default Debian user is `linux`).
+
+<a name="instance-settings"></a>
+## Instance settings
+Located at [`openstack/roles/deploy/vars/main.yaml`](openstack/roles/deploy/vars/main.yaml).  
+Here you can customize your virtual machine specifications:
+  - Specify the region name
+  - Choose an availability zone. For Open Telekom Cloud reference see [here](https://docs.otc.t-systems.com/en-us/endpoint/index.html).
+  - Change the OS image (For T-Pot we need Debian)
+  - (Optional) Change the volume size
+  - Specify your key pair (:warning: Mandatory)
+  - (Optional) Change the instance type (flavor)  
+    `s2.medium.8` corresponds to 1 vCPU and 8GB of RAM and is the minimum required flavor.  
+    A full list of Open telekom Cloud flavors can be found [here](https://docs.otc.t-systems.com/en-us/usermanual/ecs/en-us_topic_0035470096.html).
+
+```
+region_name: eu-de
+availability_zone: eu-de-03
+image: Standard_Debian_10_latest
+volume_size: 128
+key_name: your-KeyPair
+flavor: s2.medium.8
+```
+
+<a name="user-password"></a>
+## User password
+Located at [`openstack/roles/install/vars/main.yaml`](openstack/roles/install/vars/main.yaml).  
+Here you can set the password for your Debian user (**you should definitely change that**).
+```
+user_password: LiNuXuSeRPaSs#
+```
+
+<a name="tpot-conf"></a>
+## Configure `tpot.conf.dist`
+The file is located in [`iso/installer/tpot.conf.dist`](../../iso/installer/tpot.conf.dist).  
+Here you can choose:
+  - between the various T-Pot editions
+  - a username for the web interface
+  - a password for the web interface (**you should definitely change that**)
+
+```
+# tpot configuration file
+# myCONF_TPOT_FLAVOR=[STANDARD, SENSOR, INDUSTRIAL, COLLECTOR, NEXTGEN]
+myCONF_TPOT_FLAVOR='STANDARD'
+myCONF_WEB_USER='webuser'
+myCONF_WEB_PW='w3b$ecret'
+```
+
+<a name="ews-cfg"></a>
+## Optional: Custom `ews.cfg`
+Enable this by uncommenting the role in the [deploy_tpot.yaml](openstack/deploy_tpot.yaml) playbook.
+```
+#    - custom_ews
+```
+
+You can use a custom config file for `ewsposter`.  
+e.g. when you have your own credentials for delivering data to our [Sicherheitstacho](https://sicherheitstacho.eu/start/main).  
+You can find the `ews.cfg` template file here: [`openstack/roles/custom_ews/templates/ews.cfg`](openstack/roles/custom_ews/templates/ews.cfg) and adapt it for your needs.
+
+For setting custom credentials, these settings would be relevant for you (the rest of the file can stay as is):  
+```
+[MAIN]
+...
+contact = your_email_address
+...
+
+[EWS]
+...
+username = your_username
+token = your_token
+...
+```
+
+<a name="hpfeeds"></a>
+## Optional: Custom HPFEEDS
+Enable this by uncommenting the role in the [deploy_tpot.yaml](openstack/deploy_tpot.yaml) playbook.
+```
+#    - custom_hpfeeds
+```
+
+You can specify custom HPFEEDS in [`openstack/roles/custom_hpfeeds/templates/hpfeeds.cfg`](openstack/roles/custom_hpfeeds/templates/hpfeeds.cfg).  
+That file contains the defaults (turned off) and you can adapt it for your needs, e.g. for SISSDEN:
+```
+myENABLE=true
+myHOST=hpfeeds.sissden.eu
+myPORT=10000
+myCHANNEL=t-pot.events
+myCERT=/opt/ewsposter/sissden.pem
+myIDENT=your_user
+mySECRET=your_secret
+myFORMAT=json
+```
+
+<a name="deploy"></a>
+# Deploying a T-Pot :honey_pot::honeybee:
+Now, after configuring everything, we can finally start deploying T-Pots!  
+Go to the [`openstack`](openstack) folder and run the Ansible Playbook with:  
+`ansible-playbook deploy_tpot.yaml`  
+(Yes, it is as easy as that :smile:)
+
+If you are running on a machine which asks for a sudo password, you can use:  
+`ansible-playbook --ask-become-pass deploy_tpot.yaml`
+
+The Playbook will first install required packages on the Ansible Master and then deploy a new server instance.  
+After that, T-Pot gets installed and configured on the newly created host, optionally custom configs are applied and finally it reboots.
+
+<a name="documentation"></a>
+# Further documentation
+- [Ansible Documentation](https://docs.ansible.com/ansible/latest/)
+- [Cloud modules — Ansible Documentation](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html)
+- [os_server – Create/Delete Compute Instances from OpenStack — Ansible Documentation](https://docs.ansible.com/ansible/latest/modules/os_server_module.html)
+- [Open Telekom Cloud Help Center](https://docs.otc.t-systems.com/)
+- [Open Telekom Cloud API Overview](https://docs.otc.t-systems.com/en-us/api/wp/en-us_topic_0052070394.html)

cloud/ansible/openstack/ansible.cfg

@@ -0,0 +1,6 @@
+[defaults]
+host_key_checking = false
+
+[ssh_connection]
+scp_if_ssh = true
+ssh_args = -o ServerAliveInterval=60

cloud/ansible/openstack/deploy_tpot.yaml

@@ -0,0 +1,21 @@
+- name: Check host prerequisites
+  hosts: localhost
+  become: yes
+  roles:
+    - check
+
+- name: Deploy instance
+  hosts: localhost
+  roles:
+    - deploy
+
+- name: Install T-Pot on new instance
+  hosts: TPOT
+  remote_user: linux
+  become: yes
+  gather_facts: no
+  roles:
+    - install
+#    - custom_ews
+#    - custom_hpfeeds
+    - reboot

cloud/ansible/openstack/roles/check/tasks/main.yaml

@@ -0,0 +1,17 @@
+- name: Install dependencies
+  package:
+    name:
+      - pwgen
+      - python-setuptools
+      - python-pip
+    state: present
+
+- name: Install openstacksdk
+  pip:
+    name: openstacksdk
+
+- name: Check if agent forwarding is enabled
+  fail:
+    msg: Please enable agent forwarding to allow Ansible to connect to the remote host!
+  ignore_errors: yes
+  when: lookup('env','SSH_AUTH_SOCK') == ""

cloud/ansible/openstack/roles/custom_ews/tasks/main.yaml

@@ -0,0 +1,13 @@
+- name: Copy ews configuration file
+  template:
+    src: ../templates/ews.cfg
+    dest: /data/ews/conf
+    owner: root
+    group: root
+    mode: 0644
+
+- name: Patching tpot.yml with custom ews configuration file
+  lineinfile:
+    path: /opt/tpot/etc/tpot.yml
+    insertafter: "/opt/ewsposter/ews.ip"
+    line: "     - /data/ews/conf/ews.cfg:/opt/ewsposter/ews.cfg"

cloud/ansible/openstack/roles/custom_ews/templates/ews.cfg

@@ -0,0 +1,137 @@
+[MAIN]
+homedir = /opt/ewsposter/
+spooldir = /opt/ewsposter/spool/
+logdir = /opt/ewsposter/log/
+del_malware_after_send = false
+send_malware = true
+sendlimit = 500
+contact = your_email_address
+proxy =
+ip =
+
+[EWS]
+ews = true
+username = your_username
+token = your_token
+rhost_first = https://community.sicherheitstacho.eu/ews-0.1/alert/postSimpleMessage
+rhost_second = https://community.sicherheitstacho.eu/ews-0.1/alert/postSimpleMessage
+ignorecert = false
+
+[HPFEED]
+hpfeed = %(EWS_HPFEEDS_ENABLE)s
+host = %(EWS_HPFEEDS_HOST)s
+port = %(EWS_HPFEEDS_PORT)s
+channels = %(EWS_HPFEEDS_CHANNELS)s
+ident = %(EWS_HPFEEDS_IDENT)s
+secret= %(EWS_HPFEEDS_SECRET)s
+# path/to/certificate for tls broker - or "false" for non-tls broker
+tlscert = %(EWS_HPFEEDS_TLSCERT)s
+# hpfeeds submission format: "ews" (xml) or "json"
+hpfformat = %(EWS_HPFEEDS_FORMAT)s
+
+[EWSJSON]
+json = false
+jsondir = /data/ews/json/
+
+[GLASTOPFV3]
+glastopfv3 = true
+nodeid = glastopfv3-{{ ansible_hostname }}
+sqlitedb = /data/glastopf/db/glastopf.db
+malwaredir = /data/glastopf/data/files/
+
+[GLASTOPFV2]
+glastopfv2 = false
+nodeid =
+mysqlhost =
+mysqldb =
+mysqluser =
+mysqlpw =
+malwaredir =
+
+[KIPPO]
+kippo = false
+nodeid =
+mysqlhost =
+mysqldb =
+mysqluser =
+mysqlpw =
+malwaredir =
+
+[COWRIE]
+cowrie = true
+nodeid = cowrie-{{ ansible_hostname }}
+logfile = /data/cowrie/log/cowrie.json
+
+[DIONAEA]
+dionaea = true
+nodeid = dionaea-{{ ansible_hostname }}
+malwaredir = /data/dionaea/binaries/
+sqlitedb = /data/dionaea/log/dionaea.sqlite
+
+[HONEYTRAP]
+honeytrap = true
+nodeid = honeytrap-{{ ansible_hostname }}
+newversion = true
+payloaddir = /data/honeytrap/attacks/
+attackerfile = /data/honeytrap/log/attacker.log
+
+[RDPDETECT]
+rdpdetect = false
+nodeid =
+iptableslog =
+targetip =
+
+[EMOBILITY]
+eMobility = false
+nodeid = emobility-{{ ansible_hostname }}
+logfile = /data/emobility/log/centralsystemEWS.log
+
+[CONPOT]
+conpot = true
+nodeid = conpot-{{ ansible_hostname }}
+logfile = /data/conpot/log/conpot*.json
+
+[ELASTICPOT]
+elasticpot = true
+nodeid = elasticpot-{{ ansible_hostname }}
+logfile = /data/elasticpot/log/elasticpot.log
+
+[SURICATA]
+suricata = true
+nodeid = suricata-{{ ansible_hostname }}
+logfile = /data/suricata/log/eve.json
+
+[MAILONEY]
+mailoney = true
+nodeid = mailoney-{{ ansible_hostname }}
+logfile = /data/mailoney/log/commands.log
+
+[RDPY]
+rdpy = true
+nodeid = rdpy-{{ ansible_hostname }}
+logfile = /data/rdpy/log/rdpy.log
+
+[VNCLOWPOT]
+vnclowpot = true
+nodeid = vnclowpot-{{ ansible_hostname }}
+logfile = /data/vnclowpot/log/vnclowpot.log
+
+[HERALDING]
+heralding = true
+nodeid = heralding-{{ ansible_hostname }}
+logfile = /data/heralding/log/auth.csv
+
+[CISCOASA]
+ciscoasa = true
+nodeid = ciscoasa-{{ ansible_hostname }}
+logfile = /data/ciscoasa/log/ciscoasa.log
+
+[TANNER]
+tanner = true
+nodeid = tanner-{{ ansible_hostname }}
+logfile = /data/tanner/log/tanner_report.json
+
+[GLUTTON]
+glutton = true
+nodeid = glutton-{{ ansible_hostname }}
+logfile = /data/glutton/log/glutton.log

cloud/ansible/openstack/roles/custom_hpfeeds/files/hpfeeds.cfg

@@ -0,0 +1,8 @@
+myENABLE=false
+myHOST=host
+myPORT=port
+myCHANNEL=channels
+myCERT=false
+myIDENT=user
+mySECRET=secret
+myFORMAT=json

cloud/ansible/openstack/roles/custom_hpfeeds/tasks/main.yaml

@@ -0,0 +1,12 @@
+- name: Copy hpfeeds configuration file
+  copy:
+    src: ../files/hpfeeds.cfg
+    dest: /data/ews/conf
+    owner: tpot
+    group: tpot
+    mode: 0770
+  register: config
+
+- name: Applying hpfeeds settings
+  command: /opt/tpot/bin/hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg
+  when: config.changed == true

cloud/ansible/openstack/roles/deploy/tasks/main.yaml

@@ -0,0 +1,94 @@
+- name: Create T-Pot name
+  shell: echo t-pot-ansible-$(pwgen -ns 6 -1)
+  register: tpot_name
+
+- name: Import OpenStack authentication variables
+  include_vars:
+    file: roles/deploy/vars/os_auth.yaml
+  no_log: true
+
+- name: Create security group
+  os_security_group:
+    auth:
+      auth_url: "{{ auth_url }}"
+      username: "{{ username }}"
+      password: "{{ password }}"
+      project_name: "{{ project_name }}"
+      os_user_domain_name: "{{ os_user_domain_name }}"
+    name: sg-tpot-any
+    description: tpot any-any
+
+- name: Add rules to security group
+  os_security_group_rule:
+    auth:
+      auth_url: "{{ auth_url }}"
+      username: "{{ username }}"
+      password: "{{ password }}"
+      project_name: "{{ project_name }}"
+      os_user_domain_name: "{{ os_user_domain_name }}"
+    security_group: sg-tpot-any
+    remote_ip_prefix: 0.0.0.0/0
+
+- name: Create network
+  os_network:
+    auth:
+      auth_url: "{{ auth_url }}"
+      username: "{{ username }}"
+      password: "{{ password }}"
+      project_name: "{{ project_name }}"
+      os_user_domain_name: "{{ os_user_domain_name }}"
+    name: network-tpot
+
+- name: Create subnet
+  os_subnet:
+    auth:
+      auth_url: "{{ auth_url }}"
+      username: "{{ username }}"
+      password: "{{ password }}"
+      project_name: "{{ project_name }}"
+      os_user_domain_name: "{{ os_user_domain_name }}"
+    network_name: network-tpot
+    name: subnet-tpot
+    cidr: 192.168.0.0/24
+    dns_nameservers:
+      - 1.1.1.1
+      - 8.8.8.8
+
+- name: Create router
+  os_router:
+    auth:
+      auth_url: "{{ auth_url }}"
+      username: "{{ username }}"
+      password: "{{ password }}"
+      project_name: "{{ project_name }}"
+      os_user_domain_name: "{{ os_user_domain_name }}"
+    name: router-tpot
+    interfaces:
+      - subnet-tpot
+
+- name: Launch an instance
+  os_server:
+    auth:
+      auth_url: "{{ auth_url }}"
+      username: "{{ username }}"
+      password: "{{ password }}"
+      project_name: "{{ project_name }}"
+      os_user_domain_name: "{{ os_user_domain_name }}"
+    name: "{{ tpot_name.stdout }}"
+    region_name: "{{ region_name }}"
+    availability_zone: "{{ availability_zone }}"
+    image: "{{ image }}"
+    boot_from_volume: yes
+    volume_size: "{{ volume_size }}"
+    key_name: "{{ key_name }}"
+    timeout: 200
+    flavor: "{{ flavor }}"
+    security_groups: sg-tpot-any
+    network: network-tpot
+  register: tpot
+
+- name: Add instance to inventory
+  add_host:
+    hostname: "{{ tpot_name.stdout }}"
+    ansible_host: "{{ tpot.server.public_v4 }}"
+    groups: TPOT

cloud/ansible/openstack/roles/deploy/vars/main.yaml

@@ -0,0 +1,6 @@
+region_name: eu-de
+availability_zone: eu-de-03
+image: Standard_Debian_10_latest
+volume_size: 128
+key_name: your-KeyPair
+flavor: s2.medium.8

cloud/ansible/openstack/roles/deploy/vars/os_auth.yaml

@@ -0,0 +1,5 @@
+auth_url: https://iam.eu-de.otc.t-systems.com/v3
+username: your_api_user
+password: your_password
+project_name: eu-de_your_project
+os_user_domain_name: OTC-EU-DE-000000000010000XXXXX

cloud/ansible/openstack/roles/install/tasks/main.yaml

@@ -0,0 +1,48 @@
+- name: Waiting for SSH connection
+  wait_for_connection:
+
+- name: Gathering facts
+  setup:
+
+- name: Cloning T-Pot install directory
+  git:
+    repo: "https://github.com/dtag-dev-sec/tpotce.git"
+    dest: /root/tpot
+
+- name: Prepare to set user password 
+  set_fact:
+    user_name: "{{ ansible_user }}"
+    user_salt: "s0mew1ck3dTpoT"
+  no_log: true
+
+- name: Changing password for user {{ user_name }}
+  user:
+   name: "{{ ansible_user }}"
+   password: "{{ user_password | password_hash('sha512', user_salt) }}"
+   state: present
+   shell: /bin/bash
+
+- name: Copy T-Pot configuration file
+  template:
+    src: ../../../../../../iso/installer/tpot.conf.dist
+    dest: /root/tpot.conf
+    owner: root
+    group: root
+    mode: 0644
+
+- name: Install T-Pot on instance -  be patient, this might take 15 to 30 minutes depending on the connection speed.
+  command: /root/tpot/iso/installer/install.sh --type=auto --conf=/root/tpot.conf
+
+- name: Delete T-Pot configuration file
+  file:
+    path: /root/tpot.conf
+    state: absent
+
+- name: Change unattended-upgrades to take default action
+  blockinfile:
+    dest: /etc/apt/apt.conf.d/50unattended-upgrades
+    block: |
+      Dpkg::Options {
+        "--force-confdef";
+        "--force-confold";
+      }

cloud/ansible/openstack/roles/install/vars/main.yaml

@@ -0,0 +1 @@
+user_password: LiNuXuSeRPaSs#

cloud/ansible/openstack/roles/reboot/tasks/main.yaml

@@ -0,0 +1,16 @@
+- name: Finally rebooting T-Pot
+  command: shutdown -r now
+  async: 1
+  poll: 0
+
+- name: Next login options
+  debug:
+    msg:
+    - "*****    SSH Access:"
+    - "*****    ssh {{ ansible_user }}@{{ ansible_host }} -p 64295"
+    - ""
+    - "*****    Web UI:"
+    - "*****    https://{{ ansible_host }}:64297"
+    - ""
+    - "*****    Admin UI:"
+    - "*****    https://{{ ansible_host }}:64294"

cloud/terraform/.gitignore

@@ -0,0 +1,2 @@
+**/.terraform
+**/terraform.*

cloud/terraform/README.md

@@ -0,0 +1,131 @@
+# T-Pot Terraform
+
+This [Terraform](https://www.terraform.io/) configuration can be used to provision a T-Pot instance in AWS in addition to all of the necessary pre-requisites. Specifically, the following resources will be created:
+
+* EC2 instance:
+  * t3.large (2 vCPU, 8 GiB RAM)
+  * 128GB disk
+  * [Debian Stretch](https://wiki.debian.org/Cloud/AmazonEC2Image/Stretch) (The T-Pot installation script will then upgrade this to Debian Sid)
+* AWS Security Group:
+  * TCP/UDP ports <= 64000 open to the Internet
+  * TCP ports 64294, 64295 and 64297 open to a chosen administrative IP
+
+[Cloud-init](https://cloudinit.readthedocs.io/en/latest/) is used to bootstrap the instance and install T-Pot on startup. Additional provisioning using Ansible etc. is not required.
+
+The following resources are NOT automatically created and need to be specified in the configuration below:
+
+* VPC
+* Subnet
+
+## Pre-Requisites
+
+* [Terraform](https://www.terraform.io/) 0.12
+* AWS Account
+  * Existing VPC. VPC ID should be specified in configuration below
+  * Existing subnet. Subnet ID should be specified in configuration below
+* AWS Authentication credentials should be [set using environment variables](https://www.terraform.io/docs/providers/aws/index.html#environment-variables)
+
+## Required Configuration Changes
+
+### Terraform Variables
+
+In `aws/variables.tf`, change the following variables to correspond to your existing EC2 infrastructure:
+
+* `admin_ip` - source IP address(es) that you will use to administer the system. Connections to TCP ports 64294, 64295 and 64297 will be allowed from this IP only. Multiple IPs or CIDR blocks can be specified in the format: `["127.0.0.1/32", "192.168.0.0/24"]`
+* `ec2_vpc_id`
+* `ec2_subnet_id`
+* `ec2_region`
+
+### Admin Credentials
+
+In `tpot.conf`, change the following variables:
+
+```
+myCONF_WEB_USER='webuser'
+myCONF_WEB_PW='w3b$ecret'
+```
+
+This will be used to configure credentials for the T-Pot Kibana interface. Refer to [Options](https://github.com/dtag-dev-sec/tpotce#options) for more information.
+
+## Initialising
+
+The [`terraform init`](https://www.terraform.io/docs/commands/init.html) command is used to initialize a working directory containing Terraform configuration files.
+
+```
+$ cd aws
+$ terraform init
+
+Initializing the backend...
+
+Initializing provider plugins...
+- Checking for available provider plugins...
+- Downloading plugin for provider "aws" (terraform-providers/aws) 2.16.0...
+
+The following providers do not have any version constraints in configuration,
+so the latest version was installed.
+
+To prevent automatic upgrades to new major versions that may contain breaking
+changes, it is recommended to add version = "..." constraints to the
+corresponding provider blocks in configuration, with the constraint strings
+suggested below.
+
+* provider.aws: version = "~> 2.16"
+
+Terraform has been successfully initialized!
+
+You may now begin working with Terraform. Try running "terraform plan" to see
+any changes that are required for your infrastructure. All Terraform commands
+should now work.
+
+If you ever set or change modules or backend configuration for Terraform,
+rerun this command to reinitialize your working directory. If you forget, other
+commands will detect it and remind you to do so if necessary.
+```
+
+## Applying the Configuration
+
+The [`terraform apply`](https://www.terraform.io/docs/commands/apply.html) command is used to apply the changes required to reach the desired state of the configuration, or the pre-determined set of actions generated by a [`terraform plan`](https://www.terraform.io/docs/commands/plan.html) execution plan.
+
+```
+$ terraform apply
+
+An execution plan has been generated and is shown below.
+Resource actions are indicated with the following symbols:
+  + create
+
+Terraform will perform the following actions:
+
+  # aws_instance.tpot will be created
+  ...
+
+  # aws_security_group.tpot will be created
+  ...
+
+Plan: 2 to add, 0 to change, 0 to destroy.
+
+Do you want to perform these actions?
+  Terraform will perform the actions described above.
+  Only 'yes' will be accepted to approve.
+
+  Enter a value:
+```
+
+This will perform the following actions:
+
+1. Create EC2 security group
+2. Start a Debian EC2 instance
+3. Update all packages and reboot if necessary
+4. Install T-Pot and required dependencies
+5. Reboot
+
+## Connecting to the Instance
+
+### SSH
+
+Prior to the final reboot, you will temporarily be able to SSH to port 22 as per standard. Following the reboot, port 22 is used for the honeypot. The *real* SSH server is listening on port **64295**
+
+### Browser
+
+https://www.example.com:64297/
+
+Replace with the FQDN of your EC2 instance. Refer to the [T-POT documentation](https://github.com/dtag-dev-sec/tpotce#ssh-and-web-access) for further details.

cloud/terraform/aws/main.tf

@@ -0,0 +1,66 @@
+provider "aws" {
+  region = var.ec2_region
+}
+
+resource "aws_security_group" "tpot" {
+  name        = "T-Pot"
+  description = "T-Pot Honeypot"
+  vpc_id      = var.ec2_vpc_id
+  ingress {
+    from_port   = 0
+    to_port     = 64000
+    protocol    = "tcp"
+    cidr_blocks = ["0.0.0.0/0"]
+  }
+  ingress {
+    from_port   = 0
+    to_port     = 64000
+    protocol    = "udp"
+    cidr_blocks = ["0.0.0.0/0"]
+  }
+  ingress {
+    from_port   = 64294
+    to_port     = 64294
+    protocol    = "tcp"
+    cidr_blocks = var.admin_ip
+  }
+  ingress {
+    from_port   = 64295
+    to_port     = 64295
+    protocol    = "tcp"
+    cidr_blocks = var.admin_ip
+  }
+  ingress {
+    from_port   = 64297
+    to_port     = 64297
+    protocol    = "tcp"
+    cidr_blocks = var.admin_ip
+  }
+  egress {
+    from_port   = 0
+    to_port     = 0
+    protocol    = "-1"
+    cidr_blocks = ["0.0.0.0/0"]
+  }
+  tags = {
+    Name = "T-Pot"
+  }
+}
+
+resource "aws_instance" "tpot" {
+  ami           = var.ec2_ami[var.ec2_region]
+  instance_type = var.ec2_instance_type
+  key_name      = var.ec2_ssh_key_name
+  subnet_id     = var.ec2_subnet_id
+  tags = {
+    Name = "T-Pot Honeypot"
+  }
+  root_block_device {
+    volume_type           = "gp2"
+    volume_size           = 128
+    delete_on_termination = true
+  }
+  user_data              = "${file("../cloud-init.yaml")}    content: ${base64encode(file("../tpot.conf"))}"
+  vpc_security_group_ids = [aws_security_group.tpot.id]
+  associate_public_ip_address = true
+}

cloud/terraform/aws/outputs.tf

@@ -0,0 +1,12 @@
+output "Admin_UI" {
+  value = "https://${aws_instance.tpot.public_dns}:64294/"
+}
+
+output "SSH_Access" {
+  value = "ssh -i {private_key_file} -p 64295 admin@${aws_instance.tpot.public_dns}"
+}
+
+output "Web_UI" {
+  value = "https://${aws_instance.tpot.public_dns}:64297/"
+}
+

cloud/terraform/aws/variables.tf

@@ -0,0 +1,54 @@
+variable "admin_ip" {
+  default     = ["127.0.0.1/32"]
+  description = "admin IP addresses in CIDR format"
+}
+
+variable "ec2_vpc_id" {
+  description = "ID of AWS VPC"
+  default     = "vpc-XXX"
+}
+
+variable "ec2_subnet_id" {
+  description = "ID of AWS VPC subnet"
+  default     = "subnet-YYY"
+}
+
+variable "ec2_region" {
+  description = "AWS region to launch servers"
+  default     = "eu-west-1"
+}
+
+variable "ec2_ssh_key_name" {
+  default = "default"
+}
+
+# https://aws.amazon.com/ec2/instance-types/
+# t3.large = 2 vCPU, 8 GiB RAM
+variable "ec2_instance_type" {
+  default = "t3.large"
+}
+
+# Refer to https://wiki.debian.org/Cloud/AmazonEC2Image/Buster
+variable "ec2_ami" {
+  type = map(string)
+  default = {
+    "ap-east-1"      = "ami-b7d0abc6"
+    "ap-northeast-1" = "ami-01f4f0c9374675b99"
+    "ap-northeast-2" = "ami-0855cb0c55370c38c"
+    "ap-south-1"     = "ami-00d7d1cbdcb087cf3"
+    "ap-southeast-1" = "ami-03779b1b2fbb3a9d4"
+    "ap-southeast-2" = "ami-0ce3a7c68c6b1678d"
+    "ca-central-1"   = "ami-037099906a22f210f"
+    "eu-central-1"   = "ami-0845c3902a6f2af32"
+    "eu-north-1"     = "ami-e634bf98"
+    "eu-west-1"      = "ami-06a53bf81914447b5"
+    "eu-west-2"      = "ami-053d9f0770cd2e34c"
+    "eu-west-3"      = "ami-060bf1f444f742af9"
+    "me-south-1"     = "ami-04a9a536105c72d30"
+    "sa-east-1"      = "ami-0a5fd18ed0b9c7f35"
+    "us-east-1"      = "ami-01db78123b2b99496"
+    "us-east-2"      = "ami-010ffea14ff17ebf5"
+    "us-west-1"      = "ami-0ed1af421f2a3cf40"
+    "us-west-2"      = "ami-030a304a76b181155"
+  }
+}

cloud/terraform/aws/versions.tf

@@ -0,0 +1,3 @@
+terraform {
+  required_version = ">= 0.12"
+}

cloud/terraform/cloud-init.yaml

@@ -0,0 +1,25 @@
+#cloud-config
+timezone: UTC
+
+package_update: true
+package_upgrade: true
+package_reboot_if_required: true
+
+packages:
+  - git
+
+runcmd:
+  - git clone https://github.com/dtag-dev-sec/tpotce /root/tpot
+  - /root/tpot/iso/installer/install.sh --type=auto --conf=/root/tpot.conf
+  - rm /root/tpot.conf
+  - /sbin/shutdown -r +5
+
+# The contents of tpot.conf will be base64 encoded and appended to this file
+# via the terraform configuration in main.tf
+#
+# Make sure there are no trailing new lines after "permissions" below
+write_files:
+  - encoding: b64
+    owner: root:root
+    path: /root/tpot.conf
+    permissions: '0600'

cloud/terraform/tpot.conf

@@ -0,0 +1,5 @@
+# tpot configuration file
+# myCONF_TPOT_FLAVOR=[STANDARD, SENSOR, INDUSTRIAL, COLLECTOR, NEXTGEN]
+myCONF_TPOT_FLAVOR='STANDARD'
+myCONF_WEB_USER='webuser'
+myCONF_WEB_PW='w3b$ecret'

docker/adbhoney/Dockerfile

@@ -1,31 +1,35 @@
 FROM alpine 
-
+#
+# Include dist
+ADD dist/ /root/dist/
+#
 # Install packages
-RUN apk -U --no-cache add \
+RUN apk -U add \
             git \
             libcap \
-            python \
-            python-dev && \
-
+            python3 \
+            python3-dev && \
+#
 # Install adbhoney from git
     git clone --depth=1 https://github.com/huuck/ADBHoney /opt/adbhoney && \
-    sed -i 's/dst_ip/dest_ip/' /opt/adbhoney/main.py && \
-    sed -i 's/dst_port/dest_port/' /opt/adbhoney/main.py && \
-
+    cp /root/dist/adbhoney.cfg /opt/adbhoney && \
+    sed -i 's/dst_ip/dest_ip/' /opt/adbhoney/adbhoney/core.py && \
+    sed -i 's/dst_port/dest_port/' /opt/adbhoney/adbhoney/core.py && \
+#
 # Setup user, groups and configs
     addgroup -g 2000 adbhoney && \
     adduser -S -H -s /bin/ash -u 2000 -D -g 2000 adbhoney && \
     chown -R adbhoney:adbhoney /opt/adbhoney && \
-    setcap cap_net_bind_service=+ep /usr/bin/python2.7 && \
-
+    setcap cap_net_bind_service=+ep /usr/bin/python3.7 && \
+#
 # Clean up
     apk del --purge git \
-                    python-dev && \
+                    python3-dev && \
     rm -rf /root/* && \
     rm -rf /var/cache/apk/*
-
+#
 # Set workdir and start adbhoney
 STOPSIGNAL SIGINT
 USER adbhoney:adbhoney
 WORKDIR /opt/adbhoney/
-CMD nohup /usr/bin/python main.py -l log/adbhoney.log -j log/adbhoney.json -d dl/
+CMD nohup /usr/bin/python3 run.py

docker/adbhoney/dist/adbhoney.cfg

@@ -0,0 +1,19 @@
+[honeypot]
+hostname = honeypot01
+
+address = 0.0.0.0
+port = 5555
+
+download_dir = dl/
+log_dir = log/
+
+device_id = device::http://ro.product.name =starltexx;ro.product.model=SM-G960F;ro.product.device=starlte;features=cmd,stat_v2,shell_v2
+
+[output_log]
+enabled = true
+log_file = adbhoney.log
+log_level = info
+
+[output_json]
+enabled = true
+log_file = adbhoney.json

docker/adbhoney/docker-compose.yml

@@ -14,7 +14,7 @@ services:
      - adbhoney_local
     ports:
      - "5555:5555"
-    image: "dtagdevsec/adbhoney:1811"
+    image: "dtagdevsec/adbhoney:1903"
     read_only: true
     volumes:
      - /data/adbhoney/log:/opt/adbhoney/log

docker/ciscoasa/Dockerfile

@@ -1,8 +1,8 @@
 FROM alpine
-
+#
 # Include dist
 ADD dist/ /root/dist/
-
+#
 # Setup env and apt
 RUN apk -U upgrade && \
     apk add build-base \
@@ -13,11 +13,11 @@ RUN apk -U upgrade && \
             openssl-dev \
             python3 \
             python3-dev && \
-
+#
 # Setup user
     addgroup -g 2000 ciscoasa && \
     adduser -S -s /bin/bash -u 2000 -D -g 2000 ciscoasa && \
-
+#
 # Get and install packages
     mkdir -p /opt/ && \
     cd /opt/ && \
@@ -27,7 +27,7 @@ RUN apk -U upgrade && \
     pip3 install --no-cache-dir -r requirements.txt && \
     cp /root/dist/asa_server.py /opt/ciscoasa_honeypot && \
     chown -R ciscoasa:ciscoasa /opt/ciscoasa_honeypot && \
-
+#
 # Clean up
     apk del --purge build-base \
                     git \
@@ -36,7 +36,7 @@ RUN apk -U upgrade && \
                     python3-dev && \
     rm -rf /root/* && \
     rm -rf /var/cache/apk/*
-
+#
 # Start ciscoasa
 STOPSIGNAL SIGINT
 WORKDIR /tmp/ciscoasa/

docker/ciscoasa/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/ciscoasa:1811.svg)](https://microbadger.com/images/dtagdevsec/ciscoasa:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/ciscoasa:1811.svg)](https://microbadger.com/images/dtagdevsec/ciscoasa:1811 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/ciscoasa:1903.svg)](https://microbadger.com/images/dtagdevsec/ciscoasa:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/ciscoasa:1903.svg)](https://microbadger.com/images/dtagdevsec/ciscoasa:1903 "Get your own image badge on microbadger.com")
 
 # ciscoasa
 

docker/ciscoasa/docker-compose.yml

@@ -13,7 +13,7 @@ services:
     ports:
      - "5000:5000/udp"
      - "8443:8443"
-    image: "dtagdevsec/ciscoasa:1811"
+    image: "dtagdevsec/ciscoasa:1903"
     read_only: true
     volumes:
      - /data/ciscoasa/log:/var/log/ciscoasa

docker/citrixhoneypot/Dockerfile

@@ -0,0 +1,44 @@
+FROM alpine 
+#
+# Install packages
+RUN apk -U add \
+            git \
+            libcap \
+	    openssl \
+            python3 \
+            python3-dev && \
+#
+    pip3 install --no-cache-dir python-json-logger && \
+#
+# Install CitrixHoneypot from GitHub
+#    git clone --depth=1 https://github.com/malwaretech/citrixhoneypot /opt/citrixhoneypot && \
+#    git clone --depth=1 https://github.com/vorband/CitrixHoneypot /opt/citrixhoneypot && \
+    git clone --depth=1 https://github.com/t3chn0m4g3/CitrixHoneypot /opt/citrixhoneypot && \
+#
+# Setup user, groups and configs
+    mkdir -p /opt/citrixhoneypot/logs /opt/citrixhoneypot/ssl && \
+    openssl req \
+          -nodes \
+          -x509 \
+          -newkey rsa:2048 \
+          -keyout "/opt/citrixhoneypot/ssl/key.pem" \
+          -out "/opt/citrixhoneypot/ssl/cert.pem" \
+          -days 365 \
+          -subj '/C=AU/ST=Some-State/O=Internet Widgits Pty Ltd' && \
+    addgroup -g 2000 citrixhoneypot && \
+    adduser -S -H -s /bin/ash -u 2000 -D -g 2000 citrixhoneypot && \
+    chown -R citrixhoneypot:citrixhoneypot /opt/citrixhoneypot && \
+    setcap cap_net_bind_service=+ep /usr/bin/python3.8 && \
+#
+# Clean up
+    apk del --purge git \
+                    openssl \
+                    python3-dev && \
+    rm -rf /root/* && \
+    rm -rf /var/cache/apk/*
+#
+# Set workdir and start citrixhoneypot
+STOPSIGNAL SIGINT
+USER citrixhoneypot:citrixhoneypot
+WORKDIR /opt/citrixhoneypot/
+CMD nohup /usr/bin/python3 CitrixHoneypot.py

docker/citrixhoneypot/docker-compose.yml

@@ -0,0 +1,20 @@
+version: '2.3'
+
+networks:
+  citrixhoneypot_local:
+
+services:
+
+# CitrixHoneypot service
+  citrixhoneypot:
+    build: .
+    container_name: citrixhoneypot
+    restart: always
+    networks:
+     - citrixhoneypot_local
+    ports:
+     - "443:443"
+    image: "dtagdevsec/citrixhoneypot:1903"
+    read_only: true
+    volumes:
+     - /data/citrixhoneypot/logs:/opt/citrixhoneypot/logs

docker/conpot/Dockerfile

@@ -1,10 +1,11 @@
-FROM alpine
-
+FROM alpine:3.10
+#
 # Include dist
 ADD dist/ /root/dist/
-
+#
 # Setup apt
-RUN apk -U add \
+RUN sed -i 's/dl-cdn/dl-2/g' /etc/apk/repositories && \
+    apk -U add \
              build-base \
              file \
              git \
@@ -21,7 +22,7 @@ RUN apk -U add \
              py-cryptography \
              tcpdump \
              wget && \
-
+#
 # Setup ConPot
     git clone --depth=1 https://github.com/mushorg/conpot /opt/conpot && \
     cd /opt/conpot/ && \
@@ -37,20 +38,20 @@ RUN apk -U add \
     sed -i 's/port="6969"/port="69"/' /opt/conpot/conpot/templates/default/tftp/tftp.xml && \ 
     sed -i 's/port="16100"/port="161"/' /opt/conpot/conpot/templates/IEC104/snmp/snmp.xml && \ 
     sed -i 's/port="6230"/port="623"/' /opt/conpot/conpot/templates/ipmi/ipmi/ipmi.xml && \ 
-    pip3 install --no-cache-dir -U pip setuptools && \
+    pip3 install --no-cache-dir -U setuptools && \
     pip3 install --no-cache-dir . && \
     cd / && \
     rm -rf /opt/conpot /tmp/* /var/tmp/* && \
-    setcap cap_net_bind_service=+ep /usr/bin/python3.6 && \
-    
+    setcap cap_net_bind_service=+ep /usr/bin/python3.7 && \
+#    
 # Get wireshark manuf db for scapy, setup configs, user, groups
     mkdir -p /etc/conpot /var/log/conpot /usr/share/wireshark && \
     wget https://github.com/wireshark/wireshark/raw/master/manuf -o /usr/share/wireshark/manuf && \
     cp /root/dist/conpot.cfg /etc/conpot/conpot.cfg && \
-    cp -R /root/dist/templates /usr/lib/python3.6/site-packages/conpot/ && \
+    cp -R /root/dist/templates /usr/lib/python3.7/site-packages/conpot/ && \
     addgroup -g 2000 conpot && \
     adduser -S -s /bin/ash -u 2000 -D -g 2000 conpot && \
-
+#
 # Clean up
     apk del --purge \
             build-base \
@@ -68,7 +69,7 @@ RUN apk -U add \
     rm -rf /root/* && \
     rm -rf /tmp/* && \
     rm -rf /var/cache/apk/*
-
+#
 # Start conpot
 STOPSIGNAL SIGINT
 USER conpot:conpot

docker/conpot/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/conpot:1811.svg)](https://microbadger.com/images/dtagdevsec/conpot:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/conpot:1811.svg)](https://microbadger.com/images/dtagdevsec/conpot:1811 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/conpot:1903.svg)](https://microbadger.com/images/dtagdevsec/conpot:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/conpot:1903.svg)](https://microbadger.com/images/dtagdevsec/conpot:1903 "Get your own image badge on microbadger.com")
 
 # conpot
 

docker/conpot/dist/conpot.cfg

@@ -3,7 +3,7 @@ sensorid = conpot
 
 [virtual_file_system]
 data_fs_url = %(CONPOT_TMP)s
-fs_url = tar:///usr/lib/python3.6/site-packages/conpot/data.tar
+fs_url = tar:///usr/lib/python3.7/site-packages/conpot/data.tar
 
 [session]
 timeout = 30

docker/conpot/docker-compose.yml

@@ -35,7 +35,7 @@ services:
      - "2121:21"
      - "44818:44818"
      - "47808:47808"
-    image: "dtagdevsec/conpot:1811"
+    image: "dtagdevsec/conpot:1903"
     read_only: true
     volumes:
      - /data/conpot/log:/var/log/conpot
@@ -58,7 +58,7 @@ services:
     ports:
 #     - "161:161"
      - "2404:2404"
-    image: "dtagdevsec/conpot:1811"
+    image: "dtagdevsec/conpot:1903"
     read_only: true
     volumes:
      - /data/conpot/log:/var/log/conpot
@@ -80,7 +80,7 @@ services:
      - conpot_local_guardian_ast
     ports:
      - "10001:10001"
-    image: "dtagdevsec/conpot:1811"
+    image: "dtagdevsec/conpot:1903"
     read_only: true
     volumes:
      - /data/conpot/log:/var/log/conpot
@@ -102,7 +102,7 @@ services:
      - conpot_local_ipmi
     ports:
      - "623:623"
-    image: "dtagdevsec/conpot:1811"
+    image: "dtagdevsec/conpot:1903"
     read_only: true
     volumes:
      - /data/conpot/log:/var/log/conpot
@@ -125,7 +125,7 @@ services:
     ports:
      - "1025:1025"
      - "50100:50100"
-    image: "dtagdevsec/conpot:1811"
+    image: "dtagdevsec/conpot:1903"
     read_only: true
     volumes:
      - /data/conpot/log:/var/log/conpot

docker/cowrie/Dockerfile

@@ -1,10 +1,11 @@
 FROM alpine
-
+#
 # Include dist
 ADD dist/ /root/dist/
-
+#
 # Get and install dependencies & packages
-RUN apk -U --no-cache add \
+RUN apk -U add \
+                       bash \
                        build-base \
                        git \
                        gmp-dev \
@@ -12,35 +13,40 @@ RUN apk -U --no-cache add \
                        libffi-dev \
                        mpc1-dev \
                        mpfr-dev \
+                       openssl \
                        openssl-dev \
-                       python \
-                       python-dev \
-                       py-mysqldb \
-                       py-pip \
-                       py-requests \
-                       py-setuptools && \
-
+                       python3 \
+                       python3-dev \
+                       py3-bcrypt \
+		       py3-mysqlclient \
+                       py3-requests \
+                       py3-setuptools && \
+#
 # Setup user
     addgroup -g 2000 cowrie && \
     adduser -S -s /bin/ash -u 2000 -D -g 2000 cowrie && \
-
-# Install cowrie from git
-    git clone --depth=1 https://github.com/micheloosterhof/cowrie /home/cowrie/cowrie/ -b v1.3.0 && \
-    cd /home/cowrie/cowrie && \
-    pip install --no-cache-dir --upgrade cffi pip && \
-    pip install --no-cache-dir --upgrade -r requirements.txt && \
-
+#
+# Install cowrie
+    mkdir -p /home/cowrie && \
+    cd /home/cowrie && \
+    git clone --depth=1 https://github.com/micheloosterhof/cowrie -b v2.0.0 && \
+    cd cowrie && \
+    mkdir -p log && \
+    pip3 install --upgrade pip && \
+    pip3 install --upgrade -r requirements.txt && \
+#
 # Setup configs
-    setcap cap_net_bind_service=+ep /usr/bin/python2.7 && \
+    export PYTHON_DIR=$(python3 --version | tr '[A-Z]' '[a-z]' | tr -d ' ' | cut -d '.' -f 1,2 ) && \
+    setcap cap_net_bind_service=+ep /usr/bin/$PYTHON_DIR && \
     cp /root/dist/cowrie.cfg /home/cowrie/cowrie/cowrie.cfg && \
-    chown cowrie:cowrie -R /home/cowrie/* /usr/lib/python2.7/site-packages/twisted/plugins && \
-
+    chown cowrie:cowrie -R /home/cowrie/* /usr/lib/$PYTHON_DIR/site-packages/twisted/plugins && \
+#
 # Start Cowrie once to prevent dropin.cache errors upon container start caused by read-only filesystem
-    su - cowrie -c "export PYTHONPATH=/home/cowrie/cowrie && \
+    su - cowrie -c "export PYTHONPATH=/home/cowrie/cowrie:/home/cowrie/cowrie/src && \
                     cd /home/cowrie/cowrie && \
                     /usr/bin/twistd --uid=2000 --gid=2000 -y cowrie.tac --pidfile cowrie.pid cowrie &" && \
     sleep 10 && \
-
+#
 # Clean up
     apk del --purge build-base \
                     git \
@@ -49,15 +55,16 @@ RUN apk -U --no-cache add \
                     libffi-dev \
                     mpc1-dev \
                     mpfr-dev \
-                    python-dev \
-                    py-mysqldb \
-                    py-pip && \
-    rm -rf /root/* && \
+                    openssl-dev \
+                    python3-dev \
+                    py3-mysqlclient && \
+    rm -rf /root/* /tmp/* && \
     rm -rf /var/cache/apk/* && \
-    rm -rf /home/cowrie/cowrie/cowrie.pid
-
+    rm -rf /home/cowrie/cowrie/cowrie.pid && \
+    unset PYTHON_DIR
+#
 # Start cowrie
-ENV PYTHONPATH /home/cowrie/cowrie
+ENV PYTHONPATH /home/cowrie/cowrie:/home/cowrie/cowrie/src
 WORKDIR /home/cowrie/cowrie
 USER cowrie:cowrie
 CMD ["/usr/bin/twistd", "--nodaemon", "-y", "cowrie.tac", "--pidfile", "/tmp/cowrie/cowrie.pid", "cowrie"]

docker/cowrie/Dockerfile.old

@@ -0,0 +1,70 @@
+FROM alpine
+
+# Include dist
+ADD dist/ /root/dist/
+
+# Get and install dependencies & packages
+RUN apk -U --no-cache add \
+                       bash \
+                       build-base \
+                       git \
+                       gmp-dev \
+                       libcap \
+                       libffi-dev \
+                       mpc1-dev \
+                       mpfr-dev \
+                       openssl \
+                       openssl-dev \
+                       python \
+                       python-dev \
+                       py-bcrypt \
+                       py-mysqldb \
+                       py-pip \
+                       py-requests \
+                       py-setuptools && \
+
+# Setup user
+    addgroup -g 2000 cowrie && \
+    adduser -S -s /bin/ash -u 2000 -D -g 2000 cowrie && \
+
+# Install cowrie
+    mkdir -p /home/cowrie && \
+    cd /home/cowrie && \
+    git clone --depth=1 https://github.com/micheloosterhof/cowrie -b 1.5.3 && \
+    cd cowrie && \
+    mkdir -p log && \
+    pip install --upgrade pip && \
+    pip install --upgrade -r requirements.txt && \
+
+# Setup configs
+    setcap cap_net_bind_service=+ep /usr/bin/python2.7 && \
+    cp /root/dist/cowrie.cfg /home/cowrie/cowrie/cowrie.cfg && \
+    chown cowrie:cowrie -R /home/cowrie/* /usr/lib/python2.7/site-packages/twisted/plugins && \
+
+# Start Cowrie once to prevent dropin.cache errors upon container start caused by read-only filesystem
+    su - cowrie -c "export PYTHONPATH=/home/cowrie/cowrie:/home/cowrie/cowrie/src && \
+                    cd /home/cowrie/cowrie && \
+                    /usr/bin/twistd --uid=2000 --gid=2000 -y cowrie.tac --pidfile cowrie.pid cowrie &" && \
+    sleep 10 && \
+
+# Clean up
+    apk del --purge build-base \
+                    git \
+                    gmp-dev \
+                    libcap \
+                    libffi-dev \
+                    mpc1-dev \
+                    mpfr-dev \
+                    openssl-dev \
+                    python-dev \
+                    py-mysqldb \
+                    py-pip && \
+    rm -rf /root/* && \
+    rm -rf /var/cache/apk/* && \
+    rm -rf /home/cowrie/cowrie/cowrie.pid
+
+# Start cowrie
+ENV PYTHONPATH /home/cowrie/cowrie:/home/cowrie/cowrie/src
+WORKDIR /home/cowrie/cowrie
+USER cowrie:cowrie
+CMD ["/usr/bin/twistd", "--nodaemon", "-y", "cowrie.tac", "--pidfile", "/tmp/cowrie/cowrie.pid", "cowrie"]

docker/cowrie/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/cowrie:1811.svg)](https://microbadger.com/images/dtagdevsec/cowrie:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/cowrie:1811.svg)](https://microbadger.com/images/dtagdevsec/cowrie:1811 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/cowrie:1903.svg)](https://microbadger.com/images/dtagdevsec/cowrie:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/cowrie:1903.svg)](https://microbadger.com/images/dtagdevsec/cowrie:1903 "Get your own image badge on microbadger.com")
 
 # cowrie
 

docker/cowrie/dist/cowrie.cfg

@@ -1,14 +1,53 @@
 [honeypot]
 hostname = ubuntu
+log_path = log
+download_path = dl
+share_path= share/cowrie
+state_path = /tmp/cowrie/data
+etc_path = etc
+contents_path = honeyfs
+txtcmds_path = txtcmds
+ttylog = true
+ttylog_path = log/tty
+interactive_timeout = 180
+authentication_timeout = 120
+backend = shell
+timezone = UTC
 report_public_ip = true
 auth_class = AuthRandom
 auth_class_parameters = 2, 5, 10
 reported_ssh_port = 22
 data_path = /tmp/cowrie/data
 
+[shell]
+filesystem = share/cowrie/fs.pickle 
+processes = share/cowrie/cmdoutput.json
+#arch = linux-x64-lsb
+arch = bsd-aarch64-lsb, bsd-aarch64-msb, bsd-bfin-msb, bsd-mips-lsb, bsd-mips-msb, bsd-mips64-lsb, bsd-mips64-msb, bsd-powepc-msb, bsd-powepc64-lsb, bsd-riscv64-lsb, bsd-sparc-msb, bsd-sparc64-msb, bsd-x32-lsb, bsd-x64-lsb, linux-aarch64-lsb, linux-aarch64-msb, linux-alpha-lsb, linux-am33-lsb, linux-arc-lsb, linux-arc-msb, linux-arm-lsb, linux-arm-msb, linux-avr32-lsb, linux-bfin-lsb, linux-c6x-lsb, linux-c6x-msb, linux-cris-lsb, linux-frv-msb, linux-h8300-msb, linux-hppa-msb, linux-hppa64-msb, linux-ia64-lsb, linux-m32r-msb, linux-m68k-msb, linux-microblaze-msb, linux-mips-lsb, linux-mips-msb, linux-mips64-lsb, linux-mips64-msb, linux-mn10300-lsb, linux-nios-lsb, linux-nios-msb, linux-powerpc-lsb, linux-powerpc-msb, linux-powerpc64-lsb, linux-powerpc64-msb, linux-riscv64-lsb, linux-s390x-msb, linux-sh-lsb, linux-sh-msb, linux-sparc-msb, linux-sparc64-msb, linux-tilegx-lsb, linux-tilegx-msb, linux-tilegx64-lsb, linux-tilegx64-msb, linux-x64-lsb, linux-x86-lsb, linux-xtensa-msb, osx-x32-lsb, osx-x64-lsb
+kernel_version = 3.2.0-4-amd64
+kernel_build_string = #1 SMP Debian 3.2.68-1+deb7u1
+hardware_platform = x86_64
+operating_system = GNU/Linux
+ssh_version = OpenSSH_7.9p1, OpenSSL 1.1.1a  20 Nov 2018
+
 [ssh]
-version = SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.2
+enabled = true
+rsa_public_key = etc/ssh_host_rsa_key.pub
+rsa_private_key = etc/ssh_host_rsa_key
+dsa_public_key = etc/ssh_host_dsa_key.pub
+dsa_private_key = etc/ssh_host_dsa_key
+#version = SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.2
+version = SSH-2.0-OpenSSH_7.9p1
+ciphers = aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc,aes192-cbc,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc
+macs = hmac-sha2-512,hmac-sha2-384,hmac-sha2-56,hmac-sha1,hmac-md5
+compression = zlib@openssh.com,zlib,none
 listen_endpoints = tcp:22:interface=0.0.0.0
+sftp_enabled = true
+forwarding = true
+forward_redirect = false
+forward_tunnel = false
+auth_none_enabled = false
+auth_keyboard_interactive_enabled = true
 
 [telnet]
 enabled = true
@@ -18,8 +57,13 @@ reported_port = 23
 [output_jsonlog]
 enabled = true
 logfile = log/cowrie.json
+epoch_timestamp = false
 
 [output_textlog]
 enabled = false
 logfile = log/cowrie-textlog.log
 format = text
+
+[output_crashreporter]
+enabled = false
+debug = false

docker/cowrie/docker-compose.yml

@@ -18,7 +18,7 @@ services:
     ports:
      - "22:22"
      - "23:23"
-    image: "dtagdevsec/cowrie:1811"
+    image: "dtagdevsec/cowrie:1903"
     read_only: true
     volumes:
      - /data/cowrie/downloads:/home/cowrie/cowrie/dl

docker/cyberchef/Dockerfile

@@ -1,7 +1,8 @@
-FROM alpine
-
+FROM alpine:3.10
+#
 # Get and install dependencies & packages
-RUN apk -U --no-cache add \
+RUN sed -i 's/dl-cdn/dl-2/g' /etc/apk/repositories && \
+    apk -U --no-cache add \
             curl \
             git \
             npm \
@@ -9,10 +10,10 @@ RUN apk -U --no-cache add \
     npm install -g grunt-cli && \
     npm install -g http-server && \
     npm install npm@latest -g && \
-
+#
 # Install CyberChef 
     cd /root && \
-    git clone https://github.com/gchq/cyberchef -b v8.20.0 --depth=1 && \
+    git clone https://github.com/gchq/cyberchef --depth=1 && \
     chown -R nobody:nobody cyberchef && \
     cd cyberchef && \
     npm install && \
@@ -20,16 +21,16 @@ RUN apk -U --no-cache add \
     mkdir -p /opt/cyberchef && \
     mv build/prod/* /opt/cyberchef && \
     cd / && \
-
+#
 # Clean up
     apk del --purge git \
                     npm && \
     rm -rf /root/* && \
     rm -rf /var/cache/apk/*
-
+#
 # Healthcheck
 HEALTHCHECK --retries=10 CMD curl -s -XGET 'http://127.0.0.1:8000'
-
+#
 # Set user, workdir and start spiderfoot
 USER nobody:nobody
 WORKDIR /opt/cyberchef

docker/cyberchef/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/cyberchef:1811.svg)](https://microbadger.com/images/dtagdevsec/cyberchef:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/cyberchef:1811.svg)](https://microbadger.com/images/dtagdevsec/cyberchef:1811 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/cyberchef:1903.svg)](https://microbadger.com/images/dtagdevsec/cyberchef:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/cyberchef:1903.svg)](https://microbadger.com/images/dtagdevsec/cyberchef:1903 "Get your own image badge on microbadger.com")
 
 # cyberchef
 

docker/cyberchef/docker-compose.yml

@@ -14,5 +14,5 @@ services:
      - cyberchef_local
     ports:
      - "127.0.0.1:64299:8000"
-    image: "dtagdevsec/cyberchef:1811"
+    image: "dtagdevsec/cyberchef:1903"
     read_only: true

docker/dionaea/Dockerfile

@@ -1,9 +1,9 @@
 FROM debian:stretch-slim
 ENV DEBIAN_FRONTEND noninteractive
-
+#
 # Include dist
 ADD dist/ /root/dist/
-
+#
 # Install dependencies and packages
 RUN apt-get update -y && \
     apt-get dist-upgrade -y && \
@@ -32,7 +32,7 @@ RUN apt-get update -y && \
 	python3-bson \
 	python3-yaml \
 	ttf-liberation && \
-
+#
 # Get and install dionaea
     git clone --depth=1 https://github.com/dinotools/dionaea -b 0.8.0 /root/dionaea/ && \
     cd /root/dionaea && \
@@ -41,17 +41,17 @@ RUN apt-get update -y && \
     cmake -DCMAKE_INSTALL_PREFIX:PATH=/opt/dionaea .. && \
     make && \
     make install && \
-
+#
 # Setup user and groups
     addgroup --gid 2000 dionaea && \
     adduser --system --no-create-home --shell /bin/bash --uid 2000 --disabled-password --disabled-login --gid 2000 dionaea && \
     setcap cap_net_bind_service=+ep /opt/dionaea/bin/dionaea && \
-
+#
 # Supply configs and set permissions
     chown -R dionaea:dionaea /opt/dionaea/var && \
     rm -rf /opt/dionaea/etc/dionaea/* && \
     mv /root/dist/etc/* /opt/dionaea/etc/dionaea/ && \
-
+#
 # Setup runtime and clean up
     apt-get purge -y \
       build-essential \
@@ -75,7 +75,7 @@ RUN apt-get update -y && \
       python3-dev \   
       python3-bson \
       python3-yaml && \ 
-
+#
     apt-get install -y \
       ca-certificates \
       python3 \
@@ -90,11 +90,11 @@ RUN apt-get update -y && \
       libpcap0.8 \
       libpython3.5 \
       libudns0 && \
-
+#
     apt-get autoremove --purge -y && \
     apt-get clean && \
     rm -rf /root/* /var/lib/apt/lists/* /tmp/* /var/tmp/*
-
+#
 # Start dionaea
 USER dionaea:dionaea
 CMD ["/opt/dionaea/bin/dionaea", "-u", "dionaea", "-g", "dionaea", "-c", "/opt/dionaea/etc/dionaea/dionaea.cfg"]

docker/dionaea/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/dionaea:1811.svg)](https://microbadger.com/images/dtagdevsec/dionaea:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/dionaea:1811.svg)](https://microbadger.com/images/dtagdevsec/dionaea:1811 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/dionaea:1903.svg)](https://microbadger.com/images/dtagdevsec/dionaea:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/dionaea:1903.svg)](https://microbadger.com/images/dtagdevsec/dionaea:1903 "Get your own image badge on microbadger.com")
 
 # dionaea
 

docker/dionaea/dist/etc/services/smb.yaml

@@ -11,9 +11,9 @@
     os_type: 4
 
      # Additional config
-    primary_domain: WORKGROUP
-    oem_domain_name: WORKGROUP
-    server_name: WIN_SRV
+    primary_domain: DACH
+    oem_domain_name: DACH
+    server_name: ADFS
 
      ## Windows 7 ##
     native_os: Windows 7 Professional 7600

docker/dionaea/docker-compose.yml

@@ -27,7 +27,7 @@ services:
      - "5060:5060/udp"
      - "5061:5061"
      - "27017:27017"
-    image: "dtagdevsec/dionaea:1811"
+    image: "dtagdevsec/dionaea:1903"
     read_only: true
     volumes:
      - /data/dionaea/roots/ftp:/opt/dionaea/var/dionaea/roots/ftp

docker/elasticpot/Dockerfile

@@ -1,8 +1,8 @@
 FROM alpine
-
+#
 # Include dist
 ADD dist/ /root/dist/
-
+#
 # Install packages
 RUN apk -U --no-cache add \
              git \
@@ -15,18 +15,18 @@ RUN apk -U --no-cache add \
     mkdir -p /opt && \
     cd /opt/ && \
     git clone --depth=1 https://github.com/schmalle/ElasticpotPY.git && \
-
+#
 # Setup user, groups and configs
     addgroup -g 2000 elasticpot && \
     adduser -S -H -s /bin/ash -u 2000 -D -g 2000 elasticpot && \
     mv /root/dist/elasticpot.cfg /opt/ElasticpotPY/ && \
     mkdir /opt/ElasticpotPY/log && \
-
+#
 # Clean up
     apk del --purge git && \
     rm -rf /root/* && \
     rm -rf /var/cache/apk/*
-
+#
 # Start elasticpot
 STOPSIGNAL SIGINT
 USER elasticpot:elasticpot

docker/elasticpot/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/elasticpot:1811.svg)](https://microbadger.com/images/dtagdevsec/elasticpot:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/elasticpot:1811.svg)](https://microbadger.com/images/dtagdevsec/elasticpot:1811 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/elasticpot:1903.svg)](https://microbadger.com/images/dtagdevsec/elasticpot:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/elasticpot:1903.svg)](https://microbadger.com/images/dtagdevsec/elasticpot:1903 "Get your own image badge on microbadger.com")
 
 # elasticpot
 

docker/elasticpot/docker-compose.yml

@@ -14,7 +14,7 @@ services:
      - elasticpot_local
     ports:
      - "9200:9200"
-    image: "dtagdevsec/elasticpot:1811"
+    image: "dtagdevsec/elasticpot:1903"
     read_only: true
     volumes:
      - /data/elasticpot/log:/opt/ElasticpotPY/log

docker/elk/README.md

@@ -1,11 +1,11 @@
 # Elasticsearch
-[![](https://images.microbadger.com/badges/version/dtagdevsec/elasticsearch:1811.svg)](https://microbadger.com/images/dtagdevsec/elasticsearch:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/elasticsearch:1811.svg)](https://microbadger.com/images/dtagdevsec/elasticsearch:1811 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/elasticsearch:1903.svg)](https://microbadger.com/images/dtagdevsec/elasticsearch:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/elasticsearch:1903.svg)](https://microbadger.com/images/dtagdevsec/elasticsearch:1903 "Get your own image badge on microbadger.com")
 
 # Logstash
-[![](https://images.microbadger.com/badges/version/dtagdevsec/logstash:1811.svg)](https://microbadger.com/images/dtagdevsec/logstash:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/logstash:1811.svg)](https://microbadger.com/images/dtagdevsec/logstash:1811 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/logstash:1903.svg)](https://microbadger.com/images/dtagdevsec/logstash:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/logstash:1903.svg)](https://microbadger.com/images/dtagdevsec/logstash:1903 "Get your own image badge on microbadger.com")
 
 # Kibana
-[![](https://images.microbadger.com/badges/version/dtagdevsec/kibana:1811.svg)](https://microbadger.com/images/dtagdevsec/kibana:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/kibana:1811.svg)](https://microbadger.com/images/dtagdevsec/kibana:1811 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/kibana:1903.svg)](https://microbadger.com/images/dtagdevsec/kibana:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/kibana:1903.svg)](https://microbadger.com/images/dtagdevsec/kibana:1903 "Get your own image badge on microbadger.com")
 
 # elk stack
 

docker/elk/docker-compose.yml

@@ -24,7 +24,7 @@ services:
     mem_limit: 4g
     ports:
      - "127.0.0.1:64298:9200"
-    image: "dtagdevsec/elasticsearch:1811"
+    image: "dtagdevsec/elasticsearch:1903"
     volumes:
      - /data:/data
 
@@ -39,7 +39,7 @@ services:
         condition: service_healthy
     ports:
      - "127.0.0.1:64296:5601"
-    image: "dtagdevsec/kibana:1811"
+    image: "dtagdevsec/kibana:1903"
 
 ## Logstash service
   logstash:
@@ -51,7 +51,7 @@ services:
         condition: service_healthy
     env_file:
      - /opt/tpot/etc/compose/elk_environment
-    image: "dtagdevsec/logstash:1811"
+    image: "dtagdevsec/logstash:1903"
     volumes:
      - /data:/data
      - /root/tpotce/docker/elk/logstash/dist/logstash.conf:/etc/logstash/conf.d/logstash.conf
@@ -66,5 +66,5 @@ services:
         condition: service_healthy
     ports:
      - "127.0.0.1:64302:9100"
-    image: "dtagdevsec/head:1811"
+    image: "dtagdevsec/head:1903"
     read_only: true

docker/elk/elasticsearch/Dockerfile

@@ -1,41 +1,44 @@
 FROM alpine
-
+#
 # Include dist
 ADD dist/ /root/dist/
-
+#
 # Setup env and apt
-RUN apk -U add \
+RUN sed -i 's/dl-cdn/dl-2/g' /etc/apk/repositories && \
+    apk -U --no-cache add \
+             aria2 \
              bash \
              curl \
-             openjdk8-jre \
-             wget && \
-
+             nss \
+             openjdk8-jre && \
+#
 # Get and install packages
     cd /root/dist/ && \
     mkdir -p /usr/share/elasticsearch/ && \
-    wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.5.4.tar.gz && \
-    tar xvfz elasticsearch-6.5.4.tar.gz --strip-components=1 -C /usr/share/elasticsearch/ && \
-
+    aria2c -s 16 -x 16 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.8.6.tar.gz && \
+    tar xvfz elasticsearch-6.8.6.tar.gz --strip-components=1 -C /usr/share/elasticsearch/ && \
+#
 # Add and move files
     cd /root/dist/ && \
     mkdir -p /usr/share/elasticsearch/config && \
     cp elasticsearch.yml /usr/share/elasticsearch/config/ && \
-
+#
 # Setup user, groups and configs
     addgroup -g 2000 elasticsearch && \
     adduser -S -H -s /bin/ash -u 2000 -D -g 2000 elasticsearch && \
     chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/ && \
     rm -rf /usr/share/elasticsearch/modules/x-pack-ml && \
-
+#
 # Clean up
-    apk del --purge wget && \
+    apk del --purge aria2 && \
     rm -rf /root/* && \
     rm -rf /tmp/* && \
     rm -rf /var/cache/apk/*
-
+#
 # Healthcheck
 HEALTHCHECK --retries=10 CMD curl -s -XGET 'http://127.0.0.1:9200/_cat/health'
-
+#
 # Start ELK
 USER elasticsearch:elasticsearch
+ENV JAVA_HOME=/usr/lib/jvm/java-1.8-openjdk
 CMD ["/usr/share/elasticsearch/bin/elasticsearch"]

docker/elk/elasticsearch/dist/elasticsearch.yml

@@ -7,3 +7,5 @@ path:
 http.host: 0.0.0.0
 http.cors.enabled: true
 http.cors.allow-origin: "*"
+discovery.zen.ping.unicast.hosts:
+   - localhost 

docker/elk/elasticsearch/docker-compose.yml

@@ -24,6 +24,6 @@ services:
     mem_limit: 2g
     ports:
      - "127.0.0.1:64298:9200"
-    image: "dtagdevsec/elasticsearch:1811"
+    image: "dtagdevsec/elasticsearch:1903"
     volumes:
      - /data:/data

docker/elk/head/Dockerfile

@@ -1,33 +1,33 @@
 FROM alpine
-
+#
 # Setup env and apt
 RUN apk -U add \
             curl \
             git \
             nodejs \
             nodejs-npm && \
-
+#
 # Get and install packages
     mkdir -p /usr/src/app/ && \
     cd /usr/src/app/ && \
     git clone --depth=1 https://github.com/mobz/elasticsearch-head . && \
     npm install http-server && \
     sed -i "s#\"http\:\/\/localhost\:9200\"#window.location.protocol \+ \'\/\/\' \+ window.location.hostname \+ \'\:\' \+ window.location.port \+ \'\/es\/\'#" /usr/src/app/_site/app.js && \
-
+#
 # Setup user, groups and configs
     addgroup -g 2000 head && \
     adduser -S -H -s /bin/ash -u 2000 -D -g 2000 head && \
     chown -R head:head /usr/src/app/ && \
-
+#
 # Clean up
     apk del --purge git && \
     rm -rf /root/* && \
     rm -rf /tmp/* && \
     rm -rf /var/cache/apk/*
-
+#
 # Healthcheck
 HEALTHCHECK --retries=10 CMD curl -s -XGET 'http://127.0.0.1:9100'
-
+#
 # Start elasticsearch-head
 USER head:head
 WORKDIR /usr/src/app

docker/elk/head/docker-compose.yml

@@ -12,5 +12,5 @@ services:
     #        condition: service_healthy
     ports:
      - "127.0.0.1:64302:9100"
-    image: "dtagdevsec/head:1811"
+    image: "dtagdevsec/head:1903"
     read_only: true

docker/elk/kibana/Dockerfile

@@ -1,24 +1,24 @@
-FROM alpine
-
+FROM node:10.15.2-alpine
+#
 # Include dist
 ADD dist/ /root/dist/
-
+#
 # Setup env and apt
-RUN apk -U add \
-            curl \
-            nodejs \
-            wget && \
-
+RUN sed -i 's/dl-cdn/dl-2/g' /etc/apk/repositories && \
+    apk -U --no-cache add \
+            aria2 \
+            curl && \
+#
 # Get and install packages
     cd /root/dist/ && \
     mkdir -p /usr/share/kibana/ && \
-    wget https://artifacts.elastic.co/downloads/kibana/kibana-6.5.4-linux-x86_64.tar.gz && \
-    tar xvfz kibana-6.5.4-linux-x86_64.tar.gz --strip-components=1 -C /usr/share/kibana/ && \
-
+    aria2c -s 16 -x 16 https://artifacts.elastic.co/downloads/kibana/kibana-6.8.6-linux-x86_64.tar.gz && \
+    tar xvfz kibana-6.8.6-linux-x86_64.tar.gz --strip-components=1 -C /usr/share/kibana/ && \
+#
 # Kibana's bundled node does not work in alpine
     rm /usr/share/kibana/node/bin/node && \
     ln -s /usr/bin/node /usr/share/kibana/node/bin/node && \
-
+#
 # Add and move files
     cd /root/dist/ && \
     cp kibana.svg /usr/share/kibana/src/ui/public/images/kibana.svg && \
@@ -26,45 +26,37 @@ RUN apk -U add \
     cp elk.ico /usr/share/kibana/src/ui/public/assets/favicons/favicon.ico && \
     cp elk.ico /usr/share/kibana/src/ui/public/assets/favicons/favicon-16x16.png && \
     cp elk.ico /usr/share/kibana/src/ui/public/assets/favicons/favicon-32x32.png && \
-    cp create_kibana_index.js /usr/share/kibana/src/core_plugins/elasticsearch/lib/ && \
-
-# Setup plugins, rebuild bundle
-    #cd /usr/share/kibana/plugins && \
-    #wget https://github.com/dlumbrer/kbn_radar/releases/download/Kibana-6.X/kbn_radar.tar.gz && \
-    #wget https://github.com/dlumbrer/kbn_network/releases/download/6.0.X-1/network_vis.tar.gz && \
-    #tar xvfz kbn_radar.tar.gz && \
-    #tar xvfz network_vis.tar.gz && \
-    #rm *.tar.gz && \
-    rm -rf /usr/share/kibana/optimize/bundles/* && \
-
+#
 # Setup user, groups and configs
     sed -i 's/#server.basePath: ""/server.basePath: "\/kibana"/' /usr/share/kibana/config/kibana.yml && \
     sed -i 's/#kibana.defaultAppId: "home"/kibana.defaultAppId: "dashboards"/' /usr/share/kibana/config/kibana.yml && \
     sed -i 's/#server.host: "localhost"/server.host: "0.0.0.0"/' /usr/share/kibana/config/kibana.yml && \
-    sed -i 's/#elasticsearch.url: "http:\/\/localhost:9200"/elasticsearch.url: "http:\/\/elasticsearch:9200"/' /usr/share/kibana/config/kibana.yml && \
+    sed -i 's/#elasticsearch.hosts: \["http:\/\/localhost:9200"\]/elasticsearch.hosts: \["http:\/\/elasticsearch:9200"\]/' /usr/share/kibana/config/kibana.yml && \
     sed -i 's/#server.rewriteBasePath: false/server.rewriteBasePath: false/' /usr/share/kibana/config/kibana.yml && \
-    sed -i "s/#005571/#e20074/g" /usr/share/kibana/src/ui/public/chrome/directives/global_nav/global_nav.less && \
-    sed -i "s/globalColorBlue/globalColorMagenta/g" /usr/share/kibana/src/ui/public/chrome/directives/global_nav/global_nav_link/global_nav_link.less && \
-    echo "@globalColorMagenta: #9E0051;" >> /usr/share/kibana/src/ui/public/styles/variables/colors.less && \
+    sed -i "s/#005571/#e20074/g" /usr/share/kibana/built_assets/css/plugins/kibana/index.css && \
+    sed -i "s/#007ba4/#9e0051/g" /usr/share/kibana/built_assets/css/plugins/kibana/index.css && \
+    sed -i "s/#00465d/#4f0028/g" /usr/share/kibana/built_assets/css/plugins/kibana/index.css && \
     echo "xpack.infra.enabled: false" >> /usr/share/kibana/config/kibana.yml && \ 
     echo "xpack.logstash.enabled: false" >> /usr/share/kibana/config/kibana.yml && \
     echo "xpack.canvas.enabled: false" >> /usr/share/kibana/config/kibana.yml && \
     echo "xpack.spaces.enabled: false" >> /usr/share/kibana/config/kibana.yml && \
     echo "xpack.apm.enabled: false" >> /usr/share/kibana/config/kibana.yml && \
+    echo "xpack.uptime.enabled: false" >> /usr/share/kibana/config/kibana.yml && \
+    rm -rf /usr/share/kibana/optimize/bundles/* && \
     /usr/share/kibana/bin/kibana --optimize && \
     addgroup -g 2000 kibana && \
     adduser -S -H -s /bin/ash -u 2000 -D -g 2000 kibana && \
     chown -R kibana:kibana /usr/share/kibana/ && \
-
+#
 # Clean up
-    apk del --purge wget && \
+    apk del --purge aria2 && \
     rm -rf /root/* && \
     rm -rf /tmp/* && \
     rm -rf /var/cache/apk/*
-
+#
 # Healthcheck
 HEALTHCHECK --retries=10 CMD curl -s -XGET 'http://127.0.0.1:5601'
-
+#
 # Start kibana
 STOPSIGNAL SIGKILL
 USER kibana:kibana

docker/elk/kibana/dist/create_kibana_index.js

@@ -1,38 +0,0 @@
-'use strict';
-
-var _setup_error = require('./setup_error');
-
-var _setup_error2 = _interopRequireDefault(_setup_error);
-
-function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { default: obj }; }
-
-module.exports = function (server, mappings) {
-  var _server$plugins$elast = server.plugins.elasticsearch.getCluster('admin');
-
-  const callWithInternalUser = _server$plugins$elast.callWithInternalUser;
-
-  const index = server.config().get('kibana.index');
-
-  function handleError(message) {
-    return function (err) {
-      throw new _setup_error2.default(server, message, err);
-    };
-  }
-
-  return callWithInternalUser('indices.create', {
-    index: index,
-    body: {
-      settings: {
-        number_of_shards: 1,
-        number_of_replicas: 0,
-        'index.mapper.dynamic': false
-      },
-      mappings
-    }
-  }).catch(handleError('Unable to create Kibana index "<%= kibana.index %>"')).then(function () {
-    return callWithInternalUser('cluster.health', {
-      waitForStatus: 'yellow',
-      index: index
-    }).catch(handleError('Waiting for Kibana index "<%= kibana.index %>" to come online failed.'));
-  });
-};

docker/elk/kibana/docker-compose.yml

@@ -12,4 +12,4 @@ services:
 #        condition: service_healthy
     ports:
      - "127.0.0.1:64296:5601"
-    image: "dtagdevsec/kibana:1811"
+    image: "dtagdevsec/kibana:1903"

docker/elk/logstash/Dockerfile

@@ -1,53 +1,56 @@
 FROM alpine
-
+#
 # Include dist
 ADD dist/ /root/dist/
-
+#
 # Setup env and apt
-RUN apk -U add \
+RUN sed -i 's/dl-cdn/dl-2/g' /etc/apk/repositories && \
+    apk -U --no-cache add \
+             aria2 \
              bash \
+             bzip2 \
 	     curl \
-             git \
              libc6-compat \
              libzmq \
-             openjdk8-jre \
-             wget && \
-
+             nss \
+             openjdk8-jre && \
+#
 # Get and install packages
-    git clone --depth=1 https://github.com/dtag-dev-sec/listbot /etc/listbot && \
+    mkdir -p /etc/listbot && \
+    cd /etc/listbot && \
+    aria2c -s16 -x 16 https://listbot.sicherheitstacho.eu/cve.yaml.bz2 && \
+    aria2c -s16 -x 16 https://listbot.sicherheitstacho.eu/iprep.yaml.bz2 && \
+    bunzip2 *.bz2 && \
     cd /root/dist/ && \
     mkdir -p /usr/share/logstash/ && \
-    wget https://artifacts.elastic.co/downloads/logstash/logstash-6.5.4.tar.gz && \
-    wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-ASN.tar.gz && \
-    tar xvfz logstash-6.5.4.tar.gz --strip-components=1 -C /usr/share/logstash/ && \
+    aria2c -s 16 -x 16 https://artifacts.elastic.co/downloads/logstash/logstash-6.8.6.tar.gz && \
+    tar xvfz logstash-6.8.6.tar.gz --strip-components=1 -C /usr/share/logstash/ && \
     /usr/share/logstash/bin/logstash-plugin install logstash-filter-translate && \
     /usr/share/logstash/bin/logstash-plugin install logstash-output-syslog && \
-    tar xvfz GeoLite2-ASN.tar.gz --strip-components=1 -C /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/ && \
-
+#
 # Add and move files
     cd /root/dist/ && \
     cp update.sh /usr/bin/ && \
     chmod u+x /usr/bin/update.sh && \
     mkdir -p /etc/logstash/conf.d && \
     cp logstash.conf /etc/logstash/conf.d/ && \
-    cp elasticsearch-template-es6x.json /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.4-java/lib/logstash/outputs/elasticsearch/ && \
-
+    cp elasticsearch-template-es6x.json /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-9.4.0-java/lib/logstash/outputs/elasticsearch/ && \
+#
 # Setup user, groups and configs
     addgroup -g 2000 logstash && \
     adduser -S -H -s /bin/bash -u 2000 -D -g 2000 logstash && \
     chown -R logstash:logstash /usr/share/logstash && \
     chown -R logstash:logstash /etc/listbot && \
     chmod 755 /usr/bin/update.sh && \
-
+#
 # Clean up
-    apk del --purge wget && \
     rm -rf /root/* && \
     rm -rf /tmp/* && \
     rm -rf /var/cache/apk/*
-
+#
 # Healthcheck
 HEALTHCHECK --retries=10 CMD curl -s -XGET 'http://127.0.0.1:9600'
-
+#
 # Start logstash
 #USER logstash:logstash
-CMD update.sh && exec /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf --java-execution
+CMD update.sh && exec /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf --config.reload.automatic --java-execution

docker/elk/logstash/dist/elasticsearch-template-es5x.json

@@ -4,7 +4,12 @@
   "settings" : {
     "index.refresh_interval" : "5s",
     "index.number_of_shards" : "1",
-    "index.number_of_replicas" : "0"
+    "index.number_of_replicas" : "0",
+        "mapping" : {
+          "total_fields" : {
+            "limit" : "2000"
+          }
+        }
   },
   "mappings" : {
     "_default_" : {

docker/elk/logstash/dist/elasticsearch-template-es6x.json

@@ -4,7 +4,8 @@
   "settings" : {
     "index.refresh_interval" : "5s",
     "index.number_of_shards" : "1",
-    "index.number_of_replicas" : "0"
+    "index.number_of_replicas" : "0",
+    "index.mapping.total_fields.limit": "2000"
   },
   "mappings" : {
     "_default_" : {

docker/elk/logstash/dist/logstash.conf

@@ -1,6 +1,13 @@
 # Input section
 input {
 
+# Fatt                                 
+  file {                                   
+    path => ["/data/fatt/log/fatt.log"]
+    codec => json     
+    type => "Fatt"
+  }  
+
 # Suricata
   file {
     path => ["/data/suricata/log/eve.json"]
@@ -29,6 +36,13 @@ input {
     type => "Ciscoasa"
   }
 
+# CitrixHoneypot
+  file {
+    path => ["/data/citrixhoneypot/logs/server.log"]
+    codec => json
+    type => "CitrixHoneypot"
+  }
+
 # Conpot
   file {
     path => ["/data/conpot/log/*.json"]
@@ -57,12 +71,6 @@ input {
     type => "ElasticPot"
   }
 
-# Glastopf
-  file {
-    path => ["/data/glastopf/log/glastopf.log"]
-    type => "Glastopf"
-  }
-
 # Glutton
   file {
     path => ["/data/glutton/log/glutton.log"]
@@ -76,6 +84,13 @@ input {
     type => "Heralding"
   }
 
+# Honeypy
+  file {
+    path => ["/data/honeypy/log/json.log"]
+    codec => json
+    type => "Honeypy"
+  }
+
 # Honeytrap
   file {
     path => ["/data/honeytrap/log/attackers.json"]
@@ -86,6 +101,7 @@ input {
 # Mailoney
   file {
     path => ["/data/mailoney/log/commands.log"]
+    codec => json
     type => "Mailoney"
   }
 
@@ -121,6 +137,27 @@ input {
 # Filter Section
 filter {
 
+
+# Fatt
+  if [type] == "Fatt" {
+    date {
+      match => [ "timestamp", "ISO8601" ]
+    }
+    mutate {
+      rename => {
+        "sourceIp" => "src_ip"
+	"destinationIp" => "dest_ip"
+	"sourcePort" => "src_port"
+	"destinationPort" => "dest_port"
+        "gquic" => "fatt_gquic"
+        "http" => "fatt_http"
+        "rdp" => "fatt_rdp"
+        "ssh" => "fatt_ssh"
+        "tls" => "fatt_tls"
+      }
+    }
+  }
+
 # Suricata
   if [type] == "Suricata" {
     date {
@@ -131,6 +168,7 @@ filter {
       field => "[alert][signature_id]"
       destination => "[alert][cve_id]"
       dictionary_path => "/etc/listbot/cve.yaml"
+#      fallback => "-"
     }
   }
 
@@ -176,6 +214,31 @@ filter {
     }
   }
 
+# CitrixHoneypot
+  if [type] == "CitrixHoneypot" {
+    grok { 
+      match => { 
+        "message" => [ "\A\(%{IPV4:src_ip:string}:%{INT:src_port:integer}\): %{JAVAMETHOD:http.http_method:string}%{SPACE}%{CISCO_REASON:fileinfo.state:string}: %{UNIXPATH:fileinfo.filename:string}", 
+	               "\A\(%{IPV4:src_ip:string}:%{INT:src_port:integer}\): %{JAVAMETHOD:http.http_method:string}%{SPACE}%{CISCO_REASON:fileinfo.state:string}: %{GREEDYDATA:payload:string}", 
+		       "\A\(%{IPV4:src_ip:string}:%{INT:src_port:integer}\): %{S3_REQUEST_LINE:msg:string} %{CISCO_REASON:fileinfo.state:string}: %{GREEDYDATA:payload:string:string}",
+		       "\A\(%{IPV4:src_ip:string}:%{INT:src_port:integer}\): %{GREEDYDATA:msg:string}" ] 
+      } 
+    }
+    date {
+      match => [ "asctime", "ISO8601" ]
+      remove_field => ["asctime"]
+      remove_field => ["message"]
+    }
+    mutate {
+      add_field => {
+        "dest_port" => "443"
+      }
+      rename => {
+        "levelname" => "level"
+      }
+    }
+  }
+  
 # Conpot
   if [type] == "ConPot" {
     date {
@@ -235,17 +298,6 @@ filter {
     }
   }
 
-# Glastopf
-  if [type] == "Glastopf" {
-    grok {
-      match => [ "message", "\A%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{NOTSPACE}%{SPACE}%{IP:src_ip}%{SPACE}%{WORD}%{SPACE}%{URIPROTO:http_method}%{SPACE}%{NOTSPACE:http_uri}%{SPACE}%{NOTSPACE}%{SPACE}%{HOSTNAME}:%{NUMBER:dest_port:integer}" ]
-    }
-    date {
-      match => [ "timestamp", "yyyy-MM-dd HH:mm:ss,SSS" ]
-      remove_field => ["timestamp"]
-    }
-  }
-
 # Glutton
   if [type] == "Glutton" {
     date {
@@ -265,6 +317,17 @@ filter {
     }
   }
 
+# Honeypy 
+  if [type] == "Honeypy" {
+    date {
+      match => [ "timestamp", "ISO8601" ]
+      remove_field => ["timestamp"]
+      remove_field => ["date"]
+      remove_field => ["time"]
+      remove_field => ["millisecond"]
+    }
+  }
+
 # Honeytrap
   if [type] == "Honeytrap" {
     date {
@@ -282,18 +345,14 @@ filter {
 
 # Mailoney
   if [type] == "Mailoney" {
-    grok {
-      match => [ "message", "\A%{NAGIOSTIME}\[%{IPV4:src_ip}:%{INT:src_port:integer}] %{GREEDYDATA:smtp_input}" ]
+    date {
+      match => [ "timestamp", "ISO8601" ]
     }
     mutate {
       add_field => {
         "dest_port" => "25"
       }
     }
-    date {
-      match => [ "nagios_epoch", "UNIX" ]
-      remove_field => ["nagios_epoch"]
-    }
   }
 
 # Medpot
@@ -354,12 +413,12 @@ if "_grokparsefailure" in [tags] { drop {} }
     geoip {
       cache_size => 10000
       source => "src_ip"
-      database => "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-City.mmdb"
+      database => "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-City.mmdb"
     }
     geoip {
       cache_size => 10000
       source => "src_ip"
-      database => "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-ASN.mmdb"
+      database => "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-ASN.mmdb"
     }
     translate {
       refresh_interval => 86400
@@ -387,7 +446,7 @@ if "_grokparsefailure" in [tags] { drop {} }
   }
 
 # Add T-Pot hostname and external IP
-  if [type] == "Adbhoney" or [type] == "Ciscoasa" or [type] == "ConPot" or [type] == "Cowrie" or [type] == "Dionaea" or [type] == "ElasticPot" or [type] == "Glastopf" or [type] == "Glutton" or [type] == "Honeytrap" or [type] == "Heralding" or [type] == "Mailoney" or [type] == "Medpot" or [type] == "P0f" or [type] == "Rdpy" or [type] == "Suricata" or [type] == "Tanner" {
+  if [type] == "Adbhoney" or [type] == "Ciscoasa" or [type] == "CitrixHoneypot" or [type] == "ConPot" or [type] == "Cowrie" or [type] == "Dionaea" or [type] == "ElasticPot" or [type] == "Fatt" or [type] == "Glutton" or [type] == "Honeytrap" or [type] == "Heralding" or [type] == "Honeypy" or [type] == "Mailoney" or [type] == "Medpot" or [type] == "P0f" or [type] == "Rdpy" or [type] == "Suricata" or [type] == "Tanner" {
     mutate {
       add_field => {
         "t-pot_ip_ext" => "${MY_EXTIP}"
@@ -406,14 +465,14 @@ output {
 #    document_type => "doc"
   }
 
-  if [type] == "Suricata" {
-      file {
-        file_mode => 0760
-        path => "/data/suricata/log/suricata_ews.log"
-      }
-  }
+  #if [type] == "Suricata" {
+  #    file {
+  #      file_mode => 0770
+  #      path => "/data/suricata/log/suricata_ews.log"
+  #    }
+  #}
   # Debug output
-  #if [type] == "XYZ" {
+  #if [type] == "CitrixHoneypot" {
   #  stdout {
   #    codec => rubydebug
   #  }

docker/elk/logstash/dist/update.sh

@@ -6,7 +6,31 @@ function fuCLEANUP {
 }
 trap fuCLEANUP EXIT
 
-# Download updated translation maps
+# Check internet availability 
+function fuCHECKINET () {
+mySITES=$1
+error=0
+for i in $mySITES;
+  do
+    curl --connect-timeout 5 -Is $i 2>&1 > /dev/null
+      if [ $? -ne 0 ];
+        then
+          let error+=1
+      fi;
+  done;
+  echo $error
+}
+
+# Check for connectivity and download latest translation maps
+myCHECK=$(fuCHECKINET "listbot.sicherheitstacho.eu")
+if [ "$myCHECK" == "0" ];
+  then
+    echo "Connection to Listbot looks good, now downloading latest translation maps."
     cd /etc/listbot 
-git pull --all --depth=1
+    aria2c -s16 -x 16 https://listbot.sicherheitstacho.eu/cve.yaml.bz2 && \
+    aria2c -s16 -x 16 https://listbot.sicherheitstacho.eu/iprep.yaml.bz2 && \
+    bunzip2 -f *.bz2
     cd /
+  else
+    echo "Cannot reach Listbot, starting Logstash without latest translation maps."
+fi

docker/elk/logstash/docker-compose.yml

@@ -12,7 +12,7 @@ services:
 #        condition: service_healthy
     env_file:
      - /opt/tpot/etc/compose/elk_environment
-    image: "dtagdevsec/logstash:1811"
+    image: "dtagdevsec/logstash:1903"
     volumes:
      - /data:/data
      - /root/tpotce/docker/elk/logstash/dist/logstash.conf:/etc/logstash/conf.d/logstash.conf

docker/ews/Dockerfile

@@ -1,53 +1,49 @@
 FROM alpine
-
+#
 # Include dist
 ADD dist/ /root/dist/
-
+#
 # Install packages
 RUN apk -U --no-cache add \
             build-base \
             git \
             libffi-dev \
-            libssl1.0 \
+            libssl1.1 \
             openssl-dev \
-            python-dev \
-            py-cffi \
-            py-ipaddress \
-            py-lxml \
-            py-mysqldb \
-            py-pip \
-            py-pysqlite \
-            py-requests \
-            py-setuptools && \
-    pip install --no-cache-dir -U pip && \
-    pip install --no-cache-dir pyOpenSSL && \
-
+	    python3 \
+            python3-dev \
+            py3-cffi \
+            py3-ipaddress \
+	    py3-lxml \
+	    py3-mysqlclient \
+            py3-requests \
+            py3-setuptools && \
+    pip3 install --no-cache-dir -U pip && \
+    pip3 install --no-cache-dir configparser hpfeeds3 pyOpenSSL xmljson && \
+#
 # Setup ewsposter
-    git clone --depth=1 https://github.com/rep/hpfeeds /opt/hpfeeds && \
-    cd /opt/hpfeeds && \
-    python setup.py install && \
-    git clone --depth=1 https://github.com/vorband/ewsposter /opt/ewsposter && \
+    git clone --depth=1 https://github.com/dtag-dev-sec/ewsposter /opt/ewsposter && \
     mkdir -p /opt/ewsposter/spool /opt/ewsposter/log && \
-
+#
 # Setup user and groups
     addgroup -g 2000 ews && \
     adduser -S -H -u 2000 -D -g 2000 ews && \
     chown -R ews:ews /opt/ewsposter && \
-
+#
 # Supply configs
     mv /root/dist/ews.cfg /opt/ewsposter/ && \
-
+    mv /root/dist/*.pem /opt/ewsposter/ && \
+#
 # Clean up
     apk del build-base \
             git \
             openssl-dev \
-            python-dev \
-            py-pip \
+            python3-dev \
             py-setuptools && \
     rm -rf /root/* && \
     rm -rf /var/cache/apk/*
-
+#
 # Run ewsposter
 STOPSIGNAL SIGINT
 USER ews:ews
-CMD sleep 10 && exec /usr/bin/python -u /opt/ewsposter/ews.py -l 60
+CMD sleep 10 && exec /usr/bin/python3 -u /opt/ewsposter/ews.py -l $(shuf -i 10-60 -n 1)

docker/ews/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/ewsposter:1811.svg)](https://microbadger.com/images/dtagdevsec/ewsposter:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/ewsposter:1811.svg)](https://microbadger.com/images/dtagdevsec/ewsposter:1811 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/ewsposter:1903.svg)](https://microbadger.com/images/dtagdevsec/ewsposter:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/ewsposter:1903.svg)](https://microbadger.com/images/dtagdevsec/ewsposter:1903 "Get your own image badge on microbadger.com")
 
 # ewsposter
 

docker/ews/dist/ews.cfg

@@ -18,12 +18,16 @@ rhost_second = https://community.sicherheitstacho.eu/ews-0.1/alert/postSimpleMes
 ignorecert = false
 
 [HPFEED]
-hpfeed = false
-host = 0.0.0.0
-port = 0
-channels = 0
-ident = 0
-secret= 0
+hpfeed = %(EWS_HPFEEDS_ENABLE)s
+host = %(EWS_HPFEEDS_HOST)s
+port = %(EWS_HPFEEDS_PORT)s
+channels = %(EWS_HPFEEDS_CHANNELS)s
+ident = %(EWS_HPFEEDS_IDENT)s
+secret= %(EWS_HPFEEDS_SECRET)s
+# path/to/certificate for tls broker - or "false" for non-tls broker
+tlscert = %(EWS_HPFEEDS_TLSCERT)s
+# hpfeeds submission format: "ews" (xml) or "json"
+hpfformat = %(EWS_HPFEEDS_FORMAT)s
 
 [EWSJSON]
 json = false
@@ -95,7 +99,7 @@ logfile = /data/elasticpot/log/elasticpot.log
 [SURICATA]
 suricata = true
 nodeid = suricata-community-01
-logfile = /data/suricata/log/suricata_ews.log
+logfile = /data/suricata/log/eve.json
 
 [MAILONEY]
 mailoney = true
@@ -126,3 +130,8 @@ logfile = /data/ciscoasa/log/ciscoasa.log
 tanner = true
 nodeid = tanner-community-01
 logfile = /data/tanner/log/tanner_report.json
+
+[GLUTTON]
+glutton = true
+nodeid = glutton-community-01
+logfile = /data/glutton/log/glutton.log

docker/ews/dist/sissden.pem

@@ -0,0 +1,70 @@
+-----BEGIN CERTIFICATE-----
+MIIGBDCCA+ygAwIBAgIBATANBgkqhkiG9w0BAQsFADCBnTEYMBYGA1UEAwwPU0lT
+U0RFTiBSb290IENBMQswCQYDVQQGEwJQTDERMA8GA1UEBwwIV2Fyc3phd2ExLjAs
+BgNVBAoMJU5hdWtvd2EgaSBBa2FkZW1pY2thIFNpZWMgS29tcHV0ZXJvd2ExEDAO
+BgNVBAsMB1NJU1NERU4xHzAdBgkqhkiG9w0BCQEWEGFkbWluQHNpc3NkZW4uZXUw
+HhcNMTcwNDExMTMxNDE2WhcNMjcwNDA5MTMxNDE2WjCBjTEbMBkGA1UEAwwSU0lT
+U0RFTiBTZXJ2aWNlIENBMQswCQYDVQQGEwJQTDEfMB0GCSqGSIb3DQEJARYQYWRt
+aW5Ac2lzc2Rlbi5ldTEuMCwGA1UECgwlTmF1a293YSBpIEFrYWRlbWlja2EgU2ll
+YyBLb21wdXRlcm93YTEQMA4GA1UECwwHU0lTU0RFTjCCAiIwDQYJKoZIhvcNAQEB
+BQADggIPADCCAgoCggIBAPFLjU6cLQoGz1s73QMPiRxYISCMUh3CXFe52Uim9a60
+nkBDLfjMFW87MNhFCcE2xmxwdPPTz4+f5+DsEV3eZf0y63NxWx+RFV+UpODuEW5n
+tWPFUDxmgKx6iAR/tyeLVNqmgtCnWzSthE0cg71dlil6onWvkMc+Wn5Kv6aXoz4e
+5YVVhNsymhhrR0BntospY8EvtPm70hHAzOty957/zixOQ/MM+4SHRsWXTlKqv0K2
+udWpkUy1Ihs3bpea2KAvn9bBWejFwy7K4q3LyhSyqwpVCYjNi+s+9z4ipSMfvAlT
+FvHrMrODv/Iz/TQOfypYSlpX2gBP9WKLgOQj3wulJnMDQlvG1XNgOAqKfEF52YGF
+eUu21UraRgDAguIIhWxRwgXenmRo8ngWjfk9Q8734PzzXt8cwzbxJWiJLMew1SiW
+I+Kg8uYNGNT4mdBeUMo92S17ZNMXVnkt1TYfxT0A0ZlTCrhXPiWITtsVZXAdqFtl
+j5hASmEcRYNgXEUQHBn13O9IinEmks2PEcqbbbKbs2Je0DS/JvxBkqES51UdsaVQ
+zITKw3deCk0pISG8WDWZ97LEeDCvAKA5l/ooKjDwfS5vWw11mTUCOdhCoF0m8Lao
+TwE1fzzNbSaqMsT6JF/n0ACabfuvF2aqCmWsZC/Hpw8LQQS62zOouCLdcqizL9+z
+AgMBAAGjXTBbMAwGA1UdEwQFMAMBAf8wCwYDVR0PBAQDAgHuMB0GA1UdDgQWBBQ4
+nurxBppBA5PTNvFFU/vhDr/NFzAfBgNVHSMEGDAWgBSDpRyQSgaBD5XvyFOA8YHH
+tbUAbzANBgkqhkiG9w0BAQsFAAOCAgEAIvA2gkYsIVH7FGuoIo9RIxgwy7G/SHNC
+Xllz6hyTx10UwbttJ+o4gdNt8WPuGnkmywFgsjL1//bFw2+fUO5IRvWKSmXzwx9N
+faRJAjQT4JNx2uOW0ctw4USngPrLjXr3UrIQQlJFtZnEyT9u5VJXX8zkhfNJudyJ
+N88YVrPEf6Gh1Q0P+yCX0rDEb3PlP2jsYyXZtcYA5kDQ6Qq7jpLT/zrjJdaPTmzh
+2NUe7jJOBfZxPCoeev7meafY2vVOgqRqMz1+DZRoOgwq+ysczzRaXmd5a2p9Tabc
+L1w5FXKNJQ4apszA0cEScI+4mBIIQ7VFT3GO098GOcYsC2MelRkgONAIyamm66AP
+tvLQAKoiK/xz3sEHN4zaZvN/YVHaSYZEXUP0QHdyL62P62a92aCNyrHpzKURhEDA
+n8cs6icxKrS4xuVa517m53zun0brjrfeltfbO7z1A2TstFYu9BHKzRuhwV9cGRHP
+EDcb7PkfA/08sDHsyfsWtzIysNo3hwCmQ6gtOW5xlrGplFfwSsXmPG4SR3ByW379
+RA5h3zzrO0g7iCvbLclqHoqLTJTMS+6U43qXjnQ7DJ+mcbhRGcMHcZVKqO3QmLm+
+mmkDNzNYfTgY52D5mXJqUK50750mQ8dwMSkD2TufSAPmAPUp90LdQ8u9CIv6gQ+x
+A08hDHJ1cdY=
+-----END CERTIFICATE-----
+-----BEGIN CERTIFICATE-----
+MIIGHDCCBASgAwIBAgIJAPZqsOOroxaHMA0GCSqGSIb3DQEBCwUAMIGdMRgwFgYD
+VQQDDA9TSVNTREVOIFJvb3QgQ0ExCzAJBgNVBAYTAlBMMREwDwYDVQQHDAhXYXJz
+emF3YTEuMCwGA1UECgwlTmF1a293YSBpIEFrYWRlbWlja2EgU2llYyBLb21wdXRl
+cm93YTEQMA4GA1UECwwHU0lTU0RFTjEfMB0GCSqGSIb3DQEJARYQYWRtaW5Ac2lz
+c2Rlbi5ldTAeFw0xNzA0MTExMzA3NTZaFw0yNzA0MDkxMzA3NTZaMIGdMRgwFgYD
+VQQDDA9TSVNTREVOIFJvb3QgQ0ExCzAJBgNVBAYTAlBMMREwDwYDVQQHDAhXYXJz
+emF3YTEuMCwGA1UECgwlTmF1a293YSBpIEFrYWRlbWlja2EgU2llYyBLb21wdXRl
+cm93YTEQMA4GA1UECwwHU0lTU0RFTjEfMB0GCSqGSIb3DQEJARYQYWRtaW5Ac2lz
+c2Rlbi5ldTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBANKT77EYYEhV
+tJUnfnvQtGttfgqIzKIV2W6nPK9aDsKRTX5BVDHF6P5ZAF1u/52ATwdyTK7+LD66
+Q/nCzyyA2kqTgdruX6VGucpD2DVVSVF6nZhV9PcISNaMXytoG2HHlqrim53E/rVa
+rskColfs7oCxama6lPKZ/rqrJlVjA1Pl5ZtxR0IORjpOyZjSbSzKQwLp/JxHPMCU
+2cVirS7aEu5UGj+Q7Ibg0AEyoAu5tnHBKun4hmIoo7LtKWNEe1TdboxOSboGJ5wd
+UTEmNH+7izZ5FAogTUINjubkf2zZ65xEnN7DT/zFS30vYU1EclqCTp96EKPANogV
+ZeBKntEN6M5azM6Q6+nFI56TV5DWHTIXm85zzeDj5JM7TQlIGTh8A5APHpr0YyUP
+AiIUrixV2lqSDrjewey5qQcWV6WbjMS72OFKh/x7+UJICJhoUw+KwnPmWSq1WAlt
+n7C+W0raSQzt7puI30LUkInKL6iEQebMoYg0eDRI5vsRIpbo+PzflIuk/Vea/D1Y
+twgRc8ujoKI9GpPJyP4yO4nY7BkShLqKJ251lEJZnxq8LiFVi8aN6ZHt//OGEtVs
+6L97cPzqFx7qx8vnyLBFk23lb8pilHK1G0nqxCCjakTruT/JgkLXnZcLu/IDSqd3
+QLjJL0rmU9q6+RTH8A782pcBUNzeLKnlAgMBAAGjXTBbMAwGA1UdEwQFMAMBAf8w
+CwYDVR0PBAQDAgHuMB0GA1UdDgQWBBSDpRyQSgaBD5XvyFOA8YHHtbUAbzAfBgNV
+HSMEGDAWgBSDpRyQSgaBD5XvyFOA8YHHtbUAbzANBgkqhkiG9w0BAQsFAAOCAgEA
+IA0U6znfPykr5PoQlXb/Wr4L5mY/ZtNAJsvJ8jwNMsj3ZlqLOJfnHHoG5LHkb2b/
+xfM1Ee2ojmYBt4VDARqrHLLbup38Ivqt0aEco3Qx/WqbIR4IlvZBF+/qKF/wIUuc
+CuBYNIy12PcLzafT+SJosj1BJ+XiUCj/RsVXIT5CxsdXIABWC+5b3T3/PrAtKk+C
+sVjA/ck1KAHDd+3VUyRjLAAekYWA9C/hek3YwWQ3OvmyHos5gxifqMMDj6bx5qgv
+AuIs4mYJlBlHE19GxRmo2TDwE0eZiUoUdavdRBbl9v7dex+AF2GegmnC1ouYc9kv
+9moNBcuPFXuJMCOCU44aTpgEKRm3QTZTvVcUza251T+4kgT2wlFyzPqQ8hcpih4t
+knlqHhNc9ibL3/qzWr093AgC9uNaNRqmqu1WAu3vs9g3DVb/RSMrUG/V0YS1GgPq
+E+nVJ1AIJoee8YaxHztRfjPsmu1R3pp633lfcRPUKCkz52dZDFRPuQP36DuJzl2M
+itTra0MtDUuRCsuJfVGe1op2wFprswLI0qy7O9N21D4Ab8g0ik+lhmpOf5DpYxmx
+C2Xpe4d/5Xlg3wIYhEs5MnfeEy4lSMA4cxwJs11gVYHba62L7/5lqzpPmHdRYHu3
+Vf0pM/6zniQpy58Pf9+9CNU15I3iWF5K3zmevFArd6s=
+-----END CERTIFICATE-----

docker/ews/docker-compose.yml

@@ -12,9 +12,19 @@ services:
     restart: always
     networks:
      - ewsposter_local
+    environment:
+     - EWS_HPFEEDS_ENABLE=false
+     - EWS_HPFEEDS_HOST=host
+     - EWS_HPFEEDS_PORT=port
+     - EWS_HPFEEDS_CHANNELS=channels
+     - EWS_HPFEEDS_IDENT=user
+     - EWS_HPFEEDS_SECRET=secret
+     - EWS_HPFEEDS_TLSCERT=/opt/ewsposter/ca.pem
+     - EWS_HPFEEDS_FORMAT=json
     env_file:
      - /opt/tpot/etc/compose/elk_environment
-    image: "dtagdevsec/ewsposter:1811"
+    image: "dtagdevsec/ewsposter:1903"
     volumes:
      - /data:/data
      - /data/ews/conf/ews.ip:/opt/ewsposter/ews.ip
+

docker/fatt/Dockerfile

@@ -0,0 +1,42 @@
+FROM alpine
+#
+# Include dist
+#ADD dist/ /root/dist/
+#
+# Get and install dependencies & packages
+RUN sed -i 's/dl-cdn/dl-2/g' /etc/apk/repositories && \
+    apk -U add \
+              git \
+              py3-libxml2 \
+              py3-lxml \
+              python3 \
+              python3-dev && \
+    apk -U add tshark --repository http://dl-3.alpinelinux.org/alpine/edge/community/ && \
+#
+# Setup user
+    addgroup -g 2000 fatt && \
+    adduser -S -s /bin/ash -u 2000 -D -g 2000 fatt && \
+#
+# Install fatt
+    mkdir -p /opt && \
+    cd /opt && \
+    git clone --depth=1 https://github.com/0x4D31/fatt && \
+    cd fatt && \
+    mkdir -p log && \
+    pip3 install --upgrade pip && \
+    pip3 install pyshark==0.4.2.2 && \
+#
+# Setup configs
+    chown fatt:fatt -R /opt/fatt/* && \
+#
+# Clean up
+    apk del --purge git \
+                    python3-dev && \
+    rm -rf /root/* && \
+    rm -rf /var/cache/apk/* 
+#
+# Start fatt
+STOPSIGNAL SIGINT
+ENV PYTHONPATH /opt/fatt
+WORKDIR /opt/fatt
+CMD python3 fatt.py -i $(/sbin/ip address | grep '^2: ' | awk '{ print $2 }' | tr -d [:punct:]) --print_output --json_logging -o log/fatt.log

docker/fatt/docker-compose.yml

@@ -0,0 +1,17 @@
+version: '2.3'
+
+services:
+
+# Fatt service
+  fatt:
+    build: .
+    container_name: fatt
+    restart: always
+    network_mode: "host"
+    cap_add:
+     - NET_ADMIN
+     - SYS_NICE
+     - NET_RAW
+    image: "dtagdevsec/fatt:1903"
+    volumes:
+     - /data/fatt/log:/opt/fatt/log