final prep for buster

Install sudo before apt-fast
@TheHADILP Nice catch :bowtie:

.github/ISSUE_TEMPLATE.md

@@ -1,38 +1,29 @@
-# Contribution
+# Issues
 
-Thank you for your decision to contribute to T-Pot.
+Before you post your issue make sure it has not been answered yet and provide `basic support information` if you come to the conclusion it is a new issue:
-
+- 🔍 Use the [search function](https://github.com/dtag-dev-sec/tpotce/issues?utf8=%E2%9C%93&q=) first
-## Issues
+- 🧐 Check our [WIKI](https://github.com/dtag-dev-sec/tpotce/wiki)
-
+- 📚 Consult the documentation of 💻 [Debian](https://www.debian.org/doc/), 🐳 [Docker](https://docs.docker.com/), the 🦌 [ELK stack](https://www.elastic.co/guide/index.html) and the 🍯 [T-Pot Readme](https://github.com/dtag-dev-sec/tpotce/blob/master/README.md).
-Please feel free to post your problems, ideas and issues [here](https://github.com/dtag-dev-sec/tpotce/issues). We will try to answer ASAP, but to speed things up we encourage you to ...
+- **⚠️ Provide [basic support information](#info) or similiar information with regard to your issue or we can not help you and will close the issue without further notice**
-- [ ] Use the [search function](https://github.com/dtag-dev-sec/tpotce/issues?utf8=%E2%9C%93&q=) first
-- [ ] Check the [FAQ](#faq)
-- [ ] Provide [basic support information](#info) with regard to your issue
-
-Thank you :smiley:
-
--
-
-<a name="faq"></a>
-### FAQ
-
-##### Where can I find the honeypot logs?
-###### The honeypot logs are located in `/data/`. You have to login via ssh and run `sudo cd /data/`. Do not change any permissions here or T-Pot will fail to work.
-
--
 
+<br>
+<br>
+<br>
 
 <a name="info"></a>
-### Baisc support information
+## ⚠️ Basic support information (commands are expected to run as `root`)
 
-- What T-Pot version are you currtently using?
+- What version of the OS are you currently using `lsb_release -a` and `uname -a`?
-- Are you running on a Intel NUC or a VM?
+- What T-Pot version are you currently using?
+- What edition (Standard, Nextgen, etc.) of T-Pot are you running?
+- What architecture are you running on (i.e. hardware, cloud, VM, etc.)?
+- Did you have any problems during the install? If yes, please attach `/install.log` `/install.err`.
 - How long has your installation been running?
-- Did you install any upgrades or packages?
+- Did you install upgrades, packages or use the update script?
-- Did you modify any scripts?
+- Did you modify any scripts or configs? If yes, please attach the changes.
-- Have you turned persistence on/off?
+- Please provide a screenshot of `glances` and `htop`.
-- How much RAM available (login via ssh and run `htop`)?
+- How much free disk space is available (`df -h`)?
-- How much stress are the CPUs under (login via ssh and run `htop`)?
+- What is the current container status (`dps.sh`)?
-- How much swap space is being used (login via ssh and run `htop`)?
+- What is the status of the T-Pot service (`systemctl status tpot`)?
-- How much free disk space is available (login via ssh and run `sudo df -h`)?
+- What ports are being occupied? Stop T-Pot `systemctl stop tpot` and run `netstat -tulpen`
-- What is the current container status (login via ssh and run `sudo start.sh`)?
+- If a single container shows as `DOWN` you can run `docker logs <container-name>` for the latest log entries

CHANGELOG.md

@@ -0,0 +1,86 @@
+# Changelog
+
+
+## 20190802
+- **Add support for Buster as base image**
+  - Install ISO is now based on Debian Buster
+  - Installation upon Debian Buster is now supported
+
+## 20190701
+- **Reworked Ansible T-Pot Deployment**
+  - Transitioned from bash script to all Ansible
+  - Reusable Ansible Playbook for OpenStack clouds
+  - Example Showcase with our Open Telekom Cloud
+  - Adaptable for other cloud providers
+
+## 20190626
+- **HPFEEDS Opt-In commandline option**
+  - Pass a hpfeeds config file as a commandline argument
+  - hpfeeds config is saved in `/data/ews/conf/hpfeeds.cfg`
+  - Update script restores hpfeeds config
+
+## 20190604
+- **Finalize Fatt support**
+  - Build visualizations, searches, dashboards
+  - Rebuild index patterns
+  - Some finishing touches
+
+## 20190601
+- **Start supporting Fatt, remove Glastopf**
+  - Build Dockerfile, Adjust logstash, installer, update and such.
+  - Glastopf is no longer supported within T-Pot
+
+## 20190528+20190531
+- **Increase total number of fields**
+  - Adjust total number of fileds for logstash templae from 1000 to 2000.
+
+## 20190526
+- **Fix build for Cowrie**
+  - Upstream changes required a new package `py-bcrypt`.
+
+## 20190525
+- **Fix build for RDPY**
+  - Building was prevented due to cache error which occurs lately on Alpine if `apk` is using `--no-ache' as options.
+
+## 20190520
+- **Adjust permissions for /data folder**
+  - Now it is possible to download files from `/data` using SCP, WINSCP or CyberDuck.
+
+## 20190513
+- **Added Ansible T-Pot Deployment on Open Telekom Cloud**
+  - Reusable Ansible Playbooks for all cloud providers
+  - Example Showcase with our Open Telekom Cloud
+
+## 20190511
+- **Add hptest script**
+  - Quickly test if the honeypots are working with `hptest.sh <[ip,host]>` based on nmap.
+
+## 20190508
+- **Add tsec / install user to tpot group**
+  - For users being able to easily download logs from the /data folder the installer now adds the `tpot` or the logged in user (`who am i`) via `usermod -a -G tpot <user>` to the tpot group. Also /data permissions will now be enforced to `770`, which is necessary for directory listings.
+
+## 20190502
+- **Fix KVPs**
+  - Some KVPs for Cowrie changed and the tagcloud was not showing any values in the Cowrie dashboard.
+  - New installations are not affected, however existing installations need to import the objects from /opt/tpot/etc/objects/kibana-objects.json.zip.
+- **Makeiso**
+  - Move to Xorriso for building the ISO image.
+  - This allows to support most of the Debian based distros, i.e. Debian, MxLinux and Ubuntu.
+
+## 20190428
+- **Rebuild ISO**
+  - The install ISO needed a rebuilt after some changes in the Debian mirrors.
+- **Disable Netselect**
+  - After some reports in the issues that some Debian mirrors were not fully synced and thus some packages were unavailable the netselect-apt feature was disabled.
+
+## 20190406
+- **Fix for SSH**
+  - In some situations the SSH Port was not written to a new line (thanks to @dpisano for reporting).
+- **Fix race condition for apt-fast**
+  - Curl and wget need to be installed before apt-fast installation.
+
+## 20190404
+- **Fix #332**
+  - If T-Pot, opposed to the requirements, does not have full internet access netselect-apt fails to determine the fastest mirror as it needs ICMP and UDP outgoing. Should netselect-apt fail the default mirrors will be used.
+- **Improve install speed with apt-fast**
+  - Migrating from a stable base install to Debian (Sid) requires downloading lots of packages. Depending on your geo location the download speed was already improved by introducing netselect-apt to determine the fastest mirror. With apt-fast the downloads will be even faster by downloading packages not only in parallel but also with multiple connections per package.

CONTRIBUTING.MD

@@ -1,38 +0,0 @@
-# Contribution
-
-Thank you for your decision to contribute to T-Pot.
-
-## Issues
-
-Please feel free to post your problems, ideas and issues [here](https://github.com/dtag-dev-sec/tpotce/issues). We will try to answer ASAP, but to speed things up we encourage you to ...
-- [ ] Use the [search function](https://github.com/dtag-dev-sec/tpotce/issues?utf8=%E2%9C%93&q=) first
-- [ ] Check the [FAQ](#faq)
-- [ ] Provide [basic support information](#info) with regard to your issue
-
-Thank you :smiley:
-
--
-
-<a name="faq"></a>
-### FAQ
-
-##### Where can I find the honeypot logs?
-###### The honeypot logs are located in `/data/`. You have to login via ssh and run `sudo su -` and then `cd /data/`. Do not change any permissions here or T-Pot will fail to work.
-
--
-
-
-<a name="info"></a>
-### Basic support information
-
-- What T-Pot version are you currtently using?
-- Are you running on a Intel NUC or a VM?
-- How long has your installation been running?
-- Did you install any upgrades or packages?
-- Did you modify any scripts?
-- Have you turned persistence on/off?
-- How much RAM available (login via ssh and run `htop`)?
-- How much stress are the CPUs under (login via ssh and run `htop`)?
-- How much swap space is being used (login via ssh and run `htop`)?
-- How much free disk space is available (login via ssh and run `sudo df -h`)?
-- What is the current container status (login via ssh and run `sudo dps.sh`)?

README.md

@@ -1,52 +1,44 @@
-# T-Pot 17.10
+# T-Pot 19.03
 
-This repository contains the necessary files to create the **[T-Pot](https://github.com/dtag-dev-sec/tpotce/releases)** ISO image.
+T-Pot 19.03 runs on Debian (Sid), is based heavily on
-The image can then be used to install T-Pot on a physical or virtual machine.
-
-In October 2016 we released
-[T-Pot 16.10](http://dtag-dev-sec.github.io/mediator/feature/2016/10/31/t-pot-16.10.html)
-
-# T-Pot 17.10
-
-T-Pot 17.10 runs on the latest 16.04 LTS Ubuntu Server Network Installer image, is based on
 
 [docker](https://www.docker.com/), [docker-compose](https://docs.docker.com/compose/)
 
 and includes dockerized versions of the following honeypots
 
+* [adbhoney](https://github.com/huuck/ADBHoney),
+* [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot),
 * [conpot](http://conpot.org/),
-* [cowrie](http://www.micheloosterhof.com/cowrie/),
+* [cowrie](https://github.com/cowrie/cowrie),
 * [dionaea](https://github.com/DinoTools/dionaea),
-* [elasticpot](https://github.com/schmalle/ElasticPot),
+* [elasticpot](https://github.com/schmalle/ElasticpotPY),
-* [emobility](https://github.com/dtag-dev-sec/emobility),
+* [glutton](https://github.com/mushorg/glutton),
-* [glastopf](http://glastopf.org/),
+* [heralding](https://github.com/johnnykv/heralding),
+* [honeypy](https://github.com/foospidy/HoneyPy),
 * [honeytrap](https://github.com/armedpot/honeytrap/),
 * [mailoney](https://github.com/awhitehatter/mailoney),
-* [rdpy](https://github.com/citronneur/rdpy) and
+* [medpot](https://github.com/schmalle/medpot),
-* [vnclowpot](https://github.com/magisterquis/vnclowpot)
+* [rdpy](https://github.com/citronneur/rdpy),
+* [snare](http://mushmush.org/),
+* [tanner](http://mushmush.org/)
 
 
 Furthermore we use the following tools
 
+* [Cockpit](https://cockpit-project.org/running) for a lightweight, webui for docker, os, real-time performance monitoring and web terminal.
+* [Cyberchef](https://gchq.github.io/CyberChef/) a web app for encryption, encoding, compression and data analysis.
 * [ELK stack](https://www.elastic.co/videos) to beautifully visualize all the events captured by T-Pot.
 * [Elasticsearch Head](https://mobz.github.io/elasticsearch-head/) a web front end for browsing and interacting with an Elastic Search cluster.
-* [Netdata](http://my-netdata.io/) for real-time performance monitoring.
+* [Fatt](https://github.com/0x4D31/fatt) a pyshark based script for extracting network metadata and fingerprints from pcap files and live network traffic.
-* [Portainer](http://portainer.io/) a web based UI for docker.
 * [Spiderfoot](https://github.com/smicallef/spiderfoot) a open source intelligence automation tool.
 * [Suricata](http://suricata-ids.org/) a Network Security Monitoring engine.
-* [Wetty](https://github.com/krishnasrinivas/wetty) a web based SSH client.
-
 
 
 # TL;DR
-1. Meet the [system requirements](#requirements). The T-Pot installation needs at least 4 GB RAM and 64 GB free disk space as well as a working internet connection.
+1. Meet the [system requirements](#requirements). The T-Pot installation needs at least 6-8 GB RAM and 128 GB free disk space as well as a working internet connection.
 2. Download the T-Pot ISO from [GitHub](https://github.com/dtag-dev-sec/tpotce/releases) or [create it yourself](#createiso).
 3. Install the system in a [VM](#vm) or on [physical hardware](#hw) with [internet access](#placement).
-4. Enjoy your favorite beverage - [watch](http://sicherheitstacho.eu/?peers=communityPeers) and [analyze](#kibana).
+4. Enjoy your favorite beverage - [watch](https://sicherheitstacho.eu) and [analyze](#kibana).
-
-Seeing is believing :bowtie:
-
-[![T-Pot 17.10](https://img.youtube.com/vi/G-_OabDowFU/0.jpg)](https://youtu.be/G-_OabDowFU)
 
 
 # Table of Contents
@@ -58,14 +50,21 @@ Seeing is believing :bowtie:
   - [Create your own ISO Image](#createiso)
   - [Running in a VM](#vm)
   - [Running on Hardware](#hardware)
+  - [Post Install User](#postinstall)
+  - [Post Install Auto](#postinstallauto)
+  - [Cloud Deployments](#cloud)
+    - [Ansible](#ansible)
+    - [Terraform](#terraform)
   - [First Run](#firstrun)
   - [System Placement](#placement)
+- [Updates](#updates)
 - [Options](#options)
   - [SSH and web access](#ssh)
   - [Kibana Dashboard](#kibana)
   - [Tools](#tools)
   - [Maintenance](#maintenance)
   - [Community Data Submission](#submission)
+  - [Opt-In HPFEEDS Data Submission](#hpfeeds-optin)
 - [Roadmap](#roadmap)
 - [Disclaimer](#disclaimer)
 - [FAQ](#faq)
@@ -73,149 +72,160 @@ Seeing is believing :bowtie:
 - [Licenses](#licenses)
 - [Credits](#credits)
 - [Stay tuned](#staytuned)
+- [Testimonial](#testimonial)
 - [Fun Fact](#funfact)
 
-<a name="background"></a>
+<a name="changelog"></a>
-# Changelog
+# Release Notes
-- **Size still matters** 😅
+- **Move from Ubuntu 18.04 to Debian (Sid)**
-	- All docker images have been rebuilt as micro containers based on Alpine Linux to even further reduce the image size and leading to image sizes (compressed) below the 50 MB mark. The uncompressed size of eMobility and the ELK stack could each be reduced by a whopping 600 MB!
+  - For almost 5 years Ubuntu LTS versions were our distributions of choice. Last year we made a design choice for T-Pot to be closer to a rolling release model and thus allowing us to issue smaller changes and releases in a more timely manner. The distribution of choice is Debian (Sid / unstable) which will provide us with the latest advancements in a Debian based distribution.
-	- A "Everything" installation now takes roughly 1.6 GB download size
+- **Include HoneyPy honeypot**
-- **docker-compose**
+  - *HoneyPy* is now included in the NEXTGEN installation type
-	- T-Pot containers are now being controlled and monitored through docker-compose and a single configuration file `/opt/tpot/etc/tpot.yml` allowing for greater flexibility and resulting in easier image management (i.e. updated images).
+- **Include Suricata 4.1.3**
-	- As a benefit only a single `systemd` script `/etc/systemd/system/tpot.service` is needed to start `systemctl start tpot` and stop `systemctl stop tpot` the T-Pot services.
+  - Building *Suricata 4.1.3* from scratch to enable JA3 and overall better protocol support.
-	- There are four pre-configured compose configurations which do reflect the T-Pot editions `/opt/tpot/etc/compose`. Simply stop the T-Pot services and copy i.e. `cp /opt/tpot/etc/compose/all.yml /opt/tpot/etc/tpot.yml`, restart the T-Pot services and the selcted edition will be running after downloading the required docker images.
+- **Update tools to the latest versions**
-- **Introducing** [Spiderfoot](https://github.com/smicallef/spiderfoot) a open source intelligence automation tool.
+  - ELK Stack 6.6.2
-- **Installation** procedure simplified
+  - CyberChef 8.27.0
-	- Within the Ubuntu Installer you only have to choose language settings
+  - SpiderFoot v3.0
-	- After the first reboot the T-Pot installer checks if internet and required services are reachable before the installation procedure begins
+  - Cockpit 188
-	- T-Pot Installer now uses a “dialog” which looks way better than the old text based installer
+  - NGINX is now built to enforce TLS 1.3 on the T-Pot WebUI
-	- `tsec` user & password dialog is now part of the T-Pot Installer
+- **Update honeypots**
-	- The self-signed certificate is now created automatically to reduce unnecessary overhead for novice users
+  - Where possible / feasible the honeypots have been updated to their latest versions.
-	- New ASCII logo and login screen pointing to web and ssh logins
+  - *Cowrie* now supports *HASSH* generated hashes which allows for an easier identification of an attacker accross IP adresses.
-	- Hostnames are now generated using an offline name generator, which still produces funny and collision free hostnames
+  - *Heralding* now supports *SOCKS5* emulation.
-- **CVE IDs for Suricata**
+- **Update Dashboards & Visualizations**
-	- Our very own [Listbot](https://github.com/dtag-dev-sec/listbot) builds translation maps for Logstash. If Logstash registers a match the events' CVE ID will be stored alongside the event within Elasticsearch.
+  - *Offset Dashboard* added to easily spot changes in attacks on a single dashboard in 24h time window.
-- **IP Reputations**
+  - *Cowrie Dashboard* modified to integrate *HASSH* support / visualizations.
-	- [Listbot](https://github.com/dtag-dev-sec/listbot) also builds translation maps for blacklisted IPs
+  - *HoneyPy Dashboard* added to support latest honeypot addition.
-	- Based upon 30+ publicly available IP blacklisting sources listbot creates a logstash translation map matching the events' source IP addresses against the IPs reputation
+  - *Suricata Dashboard* modified to integrate *JA3* support / visualizations.
-	- If the source IP is known to a blacklist service a corresponding tag will be stored with the event
+- **Debian mirror selection**
-	- Updates occur on every logstash container start; by default every 24h
+  - During base install you now have to manually select a mirror.
-- **Honeypot updates and improvements**
+  - Upon T-Pot install the mirror closest to you will be determined automatically, `netselect-apt` requires you to allow ICMP outbound.
-	- All honeypots were updated to their latest & stable versions.
+  - This solves peering problems for most of the users speeding up installation and updates.
-- **New Honeypots** were added ...
+- **Bugs**
-	* [mailoney](https://github.com/awhitehatter/mailoney)
+  - Fixed issue #298 where the import and export of objects on the shell did not work.
-		- A low interaction SMTP honeypot
+  - Fixed issue #313 where Spiderfoot raised a KeyError, which was previously fixed in upstream.
-	* [rdpy](https://github.com/citronneur/rdpy)
+  - Fixed error in Suricata where path for reference.config changed.
-		- A low interaction RDP honeypot
+- **Release Cycle**
-	* [vnclowpot](https://github.com/magisterquis/vnclowpot)
+  - As far as possible we will integrate changes now faster into the master branch, eliminating the need for monolithic releases. The update feature will be continuously improved on that behalf. However this might not account for all feature changes.
-		- A low interaction VNC honeypot
+- **HPFEEDS Opt-In**
-- **Persistence** is now enabled by default and will keep honeypot logs and tools data in `/data/` and its sub-folders by default for 30 days. You may change that behavior in `/opt/tpot/etc/logrotate/logrotate.conf`. ELK data however will be kept for 90 days by default. You may change that behavior in `/opt/tpot/etc/curator/actions.yml`. Scripts will be triggered through `/etc/crontab`.
+  - If you want to share your T-Pot data with a 3rd party HPFEEDS broker such as [SISSDEN](https://sissden.eu) you can do so by creating an account at the SISSDEN portal and run `hpfeeds_optin.sh` on T-Pot.
-- **Updates**
-	- **Docker** was updated to the latest **1.12.6** release within Ubuntu 16.04.x LTS
-	- **ELK** was updated to the latest **Kibana 5.6.3**, **Elasticsearch 5.6.3** and **Logstash 5.6.3** releases.
-	- **Suricata** was updated to the latest **4.0.0** version including the latest **Emerging Threats** community ruleset.
-- **Dashboards Makeover**
-	- We now have **160+ Visualizations** pre-configured and compiled to 14 individual **Kibana Dashboards** for every honeypot. Monitor all *honeypot events* locally on your T-Pot installation. Aside from *honeypot events* you can also view *Suricata NSM, Syslog and NGINX* events for a quick overview of local host events.
-	- View available IP reputation of any source IP address
-	- View available CVE ID for events
-	- More **Smart links** are now included.
 - **Update Feature**
-	- For the ones who like to live on the bleeding edge of T-Pot development there is now a update script available in `/opt/tpot/update.sh`. Just run the script and it will get the latest changes from the `master branch`. For now this feature is experimental and the first step to a true rolling release cycle.
+  - For the ones who like to live on the bleeding edge of T-Pot development there is now an update script available in `/opt/tpot/update.sh`.
-- **Files & Folders**
+  - This feature is beta and is mostly intended to provide you with the latest development advances without the need of reinstalling T-Pot.
-	- While the `/data` folder is still in its old place, all T-Pot relevant files and folders have been restructured and will now be installed into `/opt/tpot`. Only a few system relevant files with regard to the installed OS and its services will be copied to locations outside the T-Pot base path.
+- **Deprecated tools**
+  - *ctop* will no longer be part of T-Pot.
+- **Fix #332**
+  - If T-Pot, opposed to the requirements, does not have full internet access netselect-apt fails to determine the fastest mirror as it needs ICMP and UDP outgoing. Should netselect-apt fail the default mirrors will be used.
+- **Improve install speed with apt-fast**
+  - Migrating from a stable base install to Debian (Sid) requires downloading lots of packages. Depending on your geo location the download speed was already improved by introducing netselect-apt to determine the fastest mirror. With apt-fast the downloads will be even faster by downloading packages not only in parallel but also with multiple connections per package.
+- **HPFEEDS Opt-In commandline option**
+  - Pass a hpfeeds config file as a commandline argument
+  - hpfeeds config is saved in `/data/ews/conf/hpfeeds.cfg`
+  - Update script restores hpfeeds config
+- **Ansible T-Pot Deployment**
+  - Transitioned from bash script to all Ansible
+  - Reusable Ansible Playbook for OpenStack clouds
+  - Example Showcase with our Open Telekom Cloud
+  - Adaptable for other cloud providers
 
 <a name="concept"></a>
 # Technical Concept
 
-T-Pot is based on the network installer of Ubuntu Server 16.04.x LTS.
+T-Pot is based on the network installer Debian (Stretch). During installation the whole system will be updated to Debian (Sid).
 The honeypot daemons as well as other support components being used have been containerized using [docker](http://docker.io).
 This allows us to run multiple honeypot daemons on the same network interface while maintaining a small footprint and constrain each honeypot within its own environment.
 
-In T-Pot we combine the dockerized honeypots
+In T-Pot we combine the dockerized honeypots ...
-[conpot](http://conpot.org/),
+* [adbhoney](https://github.com/huuck/ADBHoney),
-[cowrie](http://www.micheloosterhof.com/cowrie/),
+* [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot),
-[dionaea](https://github.com/DinoTools/dionaea),
+* [conpot](http://conpot.org/),
-[elasticpot](https://github.com/schmalle/ElasticPot),
+* [cowrie](http://www.micheloosterhof.com/cowrie/),
-[emobility](https://github.com/dtag-dev-sec/emobility),
+* [dionaea](https://github.com/DinoTools/dionaea),
-[glastopf](http://glastopf.org/),
+* [elasticpot](https://github.com/schmalle/ElasticPot),
-[honeytrap](https://github.com/armedpot/honeytrap/),
+* [glutton](https://github.com/mushorg/glutton),
-[mailoney](https://github.com/awhitehatter/mailoney),
+* [heralding](https://github.com/johnnykv/heralding),
-[rdpy](https://github.com/citronneur/rdpy) and
+* [honeypy](https://github.com/foospidy/HoneyPy),
-[vnclowpot](https://github.com/magisterquis/vnclowpot) with
+* [honeytrap](https://github.com/armedpot/honeytrap/),
-[ELK stack](https://www.elastic.co/videos) to beautifully visualize all the events captured by T-Pot,
+* [mailoney](https://github.com/awhitehatter/mailoney),
-[Elasticsearch Head](https://mobz.github.io/elasticsearch-head/) a web front end for browsing and interacting with an Elastic Search cluster,
+* [medpot](https://github.com/schmalle/medpot),
-[Netdata](http://my-netdata.io/) for real-time performance monitoring,
+* [rdpy](https://github.com/citronneur/rdpy),
-[Portainer](http://portainer.io/) a web based UI for docker,
+* [snare](http://mushmush.org/),
-[Spiderfoot](https://github.com/smicallef/spiderfoot) a open source intelligence automation tool,
+* [tanner](http://mushmush.org/)
-[Suricata](http://suricata-ids.org/) a Network Security Monitoring engine and
-[Wetty](https://github.com/krishnasrinivas/wetty) a web based SSH client.
 
-![Architecture](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/doc/architecture.png)
+... with the following tools ...
+* [Cockpit](https://cockpit-project.org/running) for a lightweight, webui for docker, os, real-time performance monitoring and web terminal.
+* [Cyberchef](https://gchq.github.io/CyberChef/) a web app for encryption, encoding, compression and data analysis.
+* [ELK stack](https://www.elastic.co/videos) to beautifully visualize all the events captured by T-Pot.
+* [Elasticsearch Head](https://mobz.github.io/elasticsearch-head/) a web front end for browsing and interacting with an Elastic Search cluster.
+* [Fatt](https://github.com/0x4D31/fatt) a pyshark based script for extracting network metadata and fingerprints from pcap files and live network traffic.
+* [Spiderfoot](https://github.com/smicallef/spiderfoot) a open source intelligence automation tool.
+* [Suricata](http://suricata-ids.org/) a Network Security Monitoring engine.
 
-While data within docker containers is volatile we do now ensure a default 30 day persistence of all relevant honeypot and tool data in the well known `/data` folder and sub-folders. The persistence configuration may be adjusted in `/opt/tpot/etc/logrotate/logrotate.conf`. Once a docker container crashes, all other data produced within its environment is erased and a fresh instance is started from the corresponding docker image.<br>
+... to give you the best out-of-the-box experience possible and an easy-to-use multi-honeypot appliance.
+
+![Architecture](doc/architecture.png)
+
+While data within docker containers is volatile we do ensure a default 30 day persistence of all relevant honeypot and tool data in the well known `/data` folder and sub-folders. The persistence configuration may be adjusted in `/opt/tpot/etc/logrotate/logrotate.conf`. Once a docker container crashes, all other data produced within its environment is erased and a fresh instance is started from the corresponding docker image.<br>
 
 Basically, what happens when the system is booted up is the following:
 
 - start host system
-- start all the necessary services (i.e. docker-engine, reverse proxy, etc.)
+- start all the necessary services (i.e. cockpit, docker, etc.)
-- start all docker containers via docker-compose (honeypots, nms, elk)
+- start all docker containers via docker-compose (honeypots, nms, elk, etc.)
 
-Within the T-Pot project, we provide all the tools and documentation necessary to build your own honeypot system and contribute to our [community data view](http://sicherheitstacho.eu/?peers=communityPeers), a separate channel on our  [Sicherheitstacho](http://sicherheitstacho.eu) that is powered by T-Pot community data.
+Within the T-Pot project, we provide all the tools and documentation necessary to build your own honeypot system and contribute to our [Sicherheitstacho](https://sicherheitstacho.eu).
 
-The source code and configuration files are stored in individual GitHub repositories, which are linked below. The docker images are pre-configured for the T-Pot environment. If you want to run the docker images separately, make sure you study the docker-compose configuration (`/opt/tpot/etc/tpot.yml`) and the T-Pot systemd script (`/etc/systemd/system/tpot.service`), as they provide a good starting point for implementing changes.
+The source code and configuration files are fully stored in the T-Pot GitHub repository. The docker images are pre-configured for the T-Pot environment. If you want to run the docker images separately, make sure you study the docker-compose configuration (`/opt/tpot/etc/tpot.yml`) and the T-Pot systemd script (`/etc/systemd/system/tpot.service`), as they provide a good starting point for implementing changes.
 
-The individual docker configurations are located in the following GitHub repositories:
+The individual docker configurations are located in the [docker folder](https://github.com/dtag-dev-sec/tpotce/tree/master/docker).
-
-- [conpot](https://github.com/dtag-dev-sec/conpot)
-- [cowrie](https://github.com/dtag-dev-sec/cowrie)
-- [dionaea](https://github.com/dtag-dev-sec/dionaea)
-- [elasticpot](https://github.com/dtag-dev-sec/elasticpot)
-- [elk-stack](https://github.com/dtag-dev-sec/elk)
-- [emobility](https://github.com/dtag-dev-sec/emobility)
-- [ewsposter](https://github.com/dtag-dev-sec/ews)
-- [glastopf](https://github.com/dtag-dev-sec/glastopf)
-- [honeytrap](https://github.com/dtag-dev-sec/honeytrap)
-- [mailoney](https://github.com/dtag-dev-sec/mailoney)
-- [netdata](https://github.com/dtag-dev-sec/netdata)
-- [portainer](https://github.com/dtag-dev-sec/ui-for-docker)
-- [rdpy](https://github.com/dtag-dev-sec/rdpy)
-- [spiderfoot](https://github.com/dtag-dev-sec/spiderfoot)
-- [suricata & p0f](https://github.com/dtag-dev-sec/suricata)
-- [vnclowpot](https://github.com/dtag-dev-sec/vnclowpot)
 
 <a name="requirements"></a>
 # System Requirements
 Depending on your installation type, whether you install on [real hardware](#hardware) or in a [virtual machine](#vm), make sure your designated T-Pot system meets the following requirements:
 
-##### T-Pot Installation (Cowrie, Dionaea, ElasticPot, Glastopf, Honeytrap, Mailoney, Rdpy, Vnclowpot, ELK, Suricata+P0f & Tools)
+##### Standard Installation
-When installing the T-Pot ISO image, make sure the target system (physical/virtual) meets the following minimum requirements:
+- Honeypots: adbhoney, ciscoasa, conpot, cowrie, dionaea, elasticpot, heralding, honeytrap, mailoney, medpot, rdpy, snare & tanner
+- Tools: cockpit, cyberchef, ELK, elasticsearch head, ewsposter, NGINX, spiderfoot, p0f and suricata
 
-- 4 GB RAM (6-8 GB recommended)
+- 6-8 GB RAM (less RAM is possible but might introduce swapping)
-- 64 GB SSD (128 GB SSD recommended)
+- 128 GB SSD (smaller is possible but limits the capacity of storing events)
 - Network via DHCP
 - A working, non-proxied, internet connection
 
-##### Honeypot Installation (Cowrie, Dionaea, ElasticPot, Glastopf, Honeytrap, Mailoney, Rdpy, Vnclowpot)
+##### Sensor Installation
-When installing the T-Pot ISO image, make sure the target system (physical/virtual) meets the following minimum requirements:
+- Honeypots: adbhoney, ciscoasa, conpot, cowrie, dionaea, elasticpot, heralding, honeytrap, mailoney, medpot, rdpy, snare & tanner
+- Tools: cockpit
 
-- 3 GB RAM (4-6 GB recommended)
+- 6-8 GB RAM (less RAM is possible but might introduce swapping)
-- 64 GB SSD (64 GB SSD recommended)
+- 128 GB SSD (smaller is possible but limits the capacity of storing events)
 - Network via DHCP
 - A working, non-proxied, internet connection
 
-##### Industrial Installation (ConPot, eMobility, ELK, Suricata+P0f & Tools)
+##### Industrial Installation
-When installing the T-Pot ISO image, make sure the target system (physical/virtual) meets the following minimum requirements:
+- Honeypots: conpot, cowrie, heralding, medpot, rdpy
+- Tools: cockpit, cyberchef, ELK, elasticsearch head, ewsposter, NGINX, spiderfoot, p0f and suricata
 
-- 4 GB RAM (8 GB recommended)
+- 6-8 GB RAM (less RAM is possible but might introduce swapping)
-- 64 GB SSD (128 GB SSD recommended)
+- 128 GB SSD (smaller is possible but limits the capacity of storing events)
 - Network via DHCP
 - A working, non-proxied, internet connection
 
-##### Everything Installation (Everything, all of the above)
+##### Collector Installation (because sometimes all you want to do is catching credentials)
-When installing the T-Pot ISO image, make sure the target system (physical/virtual) meets the following minimum requirements:
+- Honeypots: heralding
+- Tools: cockpit, cyberchef, ELK, elasticsearch head, ewsposter, NGINX, spiderfoot, p0f and suricata
 
-- 8+ GB RAM
+- 6-8 GB RAM (less RAM is possible but might introduce swapping)
-- 128+ GB SSD
+- 128 GB SSD (smaller is possible but limits the capacity of storing events)
+- Network via DHCP
+- A working, non-proxied, internet connection
+
+##### NextGen Installation (Glutton replacing Honeytrap, HoneyPy replacing Elasticpot)
+- Honeypots: adbhoney, ciscoasa, conpot, cowrie, dionaea, glutton, heralding, honeypy, mailoney, rdpy, snare & tanner
+- Tools: cockpit, cyberchef, ELK, elasticsearch head, ewsposter, fatt, NGINX, spiderfoot, p0f and suricata
+
+- 6-8 GB RAM (less RAM is possible but might introduce swapping)
+- 128 GB SSD (smaller is possible but limits the capacity of storing events)
 - Network via DHCP
 - A working, non-proxied, internet connection
 
@@ -223,7 +233,7 @@ When installing the T-Pot ISO image, make sure the target system (physical/virtu
 # Installation
 The installation of T-Pot is straight forward and heavily depends on a working, transparent and non-proxied up and running internet connection. Otherwise the installation **will fail!**
 
-Firstly, decide if you want to download our prebuilt installation ISO image from [GitHub](https://github.com/dtag-dev-sec/tpotce/releases) ***or*** [create it yourself](#createiso).
+Firstly, decide if you want to download our prebuilt installation ISO image from [GitHub](https://github.com/dtag-dev-sec/tpotce/releases), [create it yourself](#createiso) ***or*** [post-install on an existing Debian 9.7 (Stretch)](#postinstall).
 
 Secondly, decide where you want to let the system run: [real hardware](#hardware) or in a [virtual machine](#vm)?
 
@@ -237,7 +247,7 @@ You can download the prebuilt installation image from [GitHub](https://github.co
 For transparency reasons and to give you the ability to customize your install, we provide you the [ISO Creator](https://github.com/dtag-dev-sec/tpotce) that enables you to create your own ISO installation image.
 
 **Requirements to create the ISO image:**
-- Ubuntu 16.04 LTS or newer as host system (others *may* work, but remain untested)
+- Debian 9.7 or newer as host system (others *may* work, but *remain* untested)
 - 4GB of free memory  
 - 32GB of free storage
 - A working internet connection
@@ -262,11 +272,11 @@ You may want to run T-Pot in a virtualized environment. The virtual system confi
 
 We successfully tested T-Pot with [VirtualBox](https://www.virtualbox.org) and [VMWare](http://www.vmware.com) with just little modifications to the default machine configurations.
 
-It is important to make sure you meet the [system requirements](#requirements) and assign a virtual harddisk >=64 GB, >=4 GB RAM and bridged networking to T-Pot.
+It is important to make sure you meet the [system requirements](#requirements) and assign a virtual harddisk and RAM according to the requirements while making sure networking is bridged.
 
 You need to enable promiscuous mode for the network interface for suricata and p0f to work properly. Make sure you enable it during configuration.
 
-If you want to use a wifi card as primary NIC for T-Pot, please be aware of the fact that not all network interface drivers support all wireless cards. E.g. in VirtualBox, you then have to choose the *"MT SERVER"* model of the NIC.
+If you want to use a wifi card as a primary NIC for T-Pot, please be aware of the fact that not all network interface drivers support all wireless cards. E.g. in VirtualBox, you then have to choose the *"MT SERVER"* model of the NIC.
 
 Lastly, mount the `tpot.iso` ISO to the VM and continue with the installation.<br>
 
@@ -280,82 +290,169 @@ If you decide to run T-Pot on dedicated hardware, just follow these steps:
 Whereas most CD burning tools allow you to burn from ISO images, the procedure to create a bootable USB stick from an ISO image depends on your system. There are various Windows GUI tools available, e.g. [this tip](http://www.ubuntu.com/download/desktop/create-a-usb-stick-on-windows) might help you.<br> On [Linux](http://askubuntu.com/questions/59551/how-to-burn-a-iso-to-a-usb-device) or [MacOS](http://www.ubuntu.com/download/desktop/create-a-usb-stick-on-mac-osx) you can use the tool *dd* or create the USB stick with T-Pot's [ISO Creator](https://github.com/dtag-dev-sec).
 2. Boot from the USB stick and install.
 
-*Please note*: We will ensure the compatibility with the Intel NUC platform, as we really like the form factor, looks and build quality.
+*Please note*: While we are performing limited tests with the Intel NUC platform other hardware platforms **remain untested**. We can not provide hardware support of any kind.
+
+<a name="postinstall"></a>
+## Post-Install User
+In some cases it is necessary to install Debian 9.7 (Stretch) on your own:
+ - Cloud provider does not offer mounting ISO images.
+ - Hardware setup needs special drivers and / or kernels.
+ - Within your company you have to setup special policies, software etc.
+ - You just like to stay on top of things.
+
+The T-Pot Universal Installer will upgrade the system to Debian (Sid) and install all required T-Pot dependencies.
+
+Just follow these steps:
+
+```
+git clone https://github.com/dtag-dev-sec/tpotce
+cd tpotce/iso/installer/
+./install.sh --type=user
+```
+
+The installer will now start and guide you through the install process.
+
+<a name="postinstallauto"></a>
+## Post-Install Auto
+You can also let the installer run automatically if you provide your own `tpot.conf`. An example is available in `tpotce/iso/installer/tpot.conf.dist`. This should make things easier in case you want to automate the installation i.e. with **Ansible**.
+
+Just follow these steps while adjusting `tpot.conf` to your needs:
+
+```
+git clone https://github.com/dtag-dev-sec/tpotce
+cd tpotce/iso/installer/
+cp tpot.conf.dist tpot.conf
+./install.sh --type=auto --conf=tpot.conf
+```
+
+The installer will start automatically and guide you through the install process.
+
+<a name="cloud"></a>
+## Cloud Deployments
+Located in the [`cloud`](cloud) folder.  
+Currently there are examples with Ansible & Terraform.  
+If you would like to contribute, you can add other cloud deployments like Chef or Puppet or extend current methods with other cloud providers.
+
+<a name="ansible"></a>
+### Ansible Deployment
+You can find an [Ansible](https://www.ansible.com/) based T-Pot deployment in the [`cloud/ansible`](cloud/ansible) folder.  
+The Playbook in the [`cloud/ansible/openstack`](cloud/ansible/openstack) folder is reusable for all OpenStack clouds out of the box.
+
+It first creates a new server and then installs and configures T-Pot.
+
+You can have a look at the Playbook and easily adapt the deploy role for other [cloud providers](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html).
+
+<a name="terraform"></a>
+### Terraform Configuration
+
+You can find [Terraform](https://www.terraform.io/) configuration in the [`cloud/terraform`](cloud/terraform) folder.
+
+This can be used to launch a virtual machine, bootstrap any dependencies and install T-Pot in a single step.
+
+Configuration for Amazon Web Services (AWS) is currently included and this can easily be extended to support other [Terraform providers](https://www.terraform.io/docs/providers/index.html).
 
 <a name="firstrun"></a>
 ## First Run
-The installation requires very little interaction, only a locale and keyboard setting has to be answered for the basic linux installation. The system will reboot and please maintain an active internet connection. The T-Pot installer will start and ask you for an installation type, password for the **tsec** user and credentials for a **web user**. Everything else will be configured automatically. All docker images and other componenents will be downloaded. Depending on your network connection and the chosen installation type, the installation may take some time. During our tests (50Mbit down, 10Mbit up), the installation is usually finished within a 30 minute timeframe.
+The installation requires very little interaction, only a locale and keyboard setting have to be answered for the basic linux installation. The system will reboot and please maintain the active internet connection. The T-Pot installer will start and ask you for an installation type, password for the **tsec** user and credentials for a **web user**. Everything else will be configured automatically. All docker images and other componenents will be downloaded. Depending on your network connection and the chosen installation type, the installation may take some time. During our tests (250Mbit down, 40Mbit up), the installation was usually finished within a 15-30 minute timeframe.
 
-Once the installation is finished, the system will automatically reboot and you will be presented with the T-Pot login screen. On the console you may login with the **tsec** user:
+Once the installation is finished, the system will automatically reboot and you will be presented with the T-Pot login screen. On the console you may login with:
 
-- user: **tsec**
+- user: **[tsec or user]** *you chose during one of the post install methods*
-- pass: **password you chose during the installation**
+- pass: **[password]** *you chose during the installation*
 
 All honeypot services are preconfigured and are starting automatically.
 
-You can also login from your browser: ``https://<your.ip>:64297``
+You can login from your browser and access the Admin UI: `https://<your.ip>:64294` or via SSH to access the command line: `ssh -l tsec -p 64295 <your.ip>`
 
-- user: **user you chose during the installation**
+- user: **[tsec or user]** *you chose during one of the post install methods*
-- pass: **password you chose during the installation**
+- pass: **[password]** *you chose during the installation*
+
+You can also login from your browser and access the Web UI: `https://<your.ip>:64297`
+- user: **[user]** *you chose during the installation*
+- pass: **[password]** *you chose during the installation*
 
 
 <a name="placement"></a>
 # System Placement
-Make sure your system is reachable through the internet. Otherwise it will not capture any attacks, other than the ones from your  internal network! We recommend you put it in an unfiltered zone, where all TCP and UDP traffic is forwarded to T-Pot's network interface.
+Make sure your system is reachable through a network you suspect intruders in / from (i.e. the internet). Otherwise T-Pot will most likely not capture any attacks, other than the ones from your  internal network! We recommend you put it in an unfiltered zone, where all TCP and UDP traffic is forwarded to T-Pot's network interface. However to avoid fingerprinting you can put T-Pot behind a firewall and forward all TCP / UDP traffic in the port range of 1-64000 to T-Pot while allowing access to ports > 64000 only from trusted IPs.
 
 A list of all relevant ports is available as part of the [Technical Concept](#concept)
 <br>
 
 Basically, you can forward as many TCP ports as you want, as honeytrap dynamically binds any TCP port that is not covered by the other honeypot daemons.
 
+In case you need external Admin UI access, forward TCP port 64294 to T-Pot, see below.
 In case you need external SSH access, forward TCP port 64295 to T-Pot, see below.
-In case you need external web access, forward TCP port 64297 to T-Pot, see below.
+In case you need external Web UI access, forward TCP port 64297 to T-Pot, see below.
 
-T-Pot requires outgoing git, http, https connections for updates (Ubuntu, Docker, GitHub, PyPi) and attack submission (ewsposter, hpfeeds). Ports and availability may vary based on your geographical location.
+T-Pot requires outgoing git, http, https connections for updates (Debian, Docker, GitHub, PyPi) and attack submission (ewsposter, hpfeeds). Ports and availability may vary based on your geographical location.
+
+<a name="updates"></a>
+# Updates
+For the ones of you who want to live on the bleeding edge of T-Pot development we introduced an update feature which will allow you to update all T-Pot relevant files to be up to date with the T-Pot master branch.
+**If you made any relevant changes to the T-Pot relevant config files make sure to create a backup first.**
+
+The Update script will:
+ - **mercilessly** overwrite local changes to be in sync with the T-Pot master branch
+ - upgrade the system to the packages available in Debian (Sid)
+ - update all resources to be in-sync with the T-Pot master branch
+ - ensure all T-Pot relevant system files will be patched / copied into the original T-Pot state
+ - restore your custom ews.cfg and HPFEED settings from `/data/ews/conf`
+
+You simply run the update script:
+```
+sudo su -
+cd /opt/tpot/
+./update.sh -y
+```
+
+**Despite all our efforts please be reminded that updates sometimes may have unforeseen consequences. Please create a backup of the machine or the files with the most value to your work.**  
 
 <a name="options"></a>
 # Options
-The system is designed to run without any interaction or maintenance and automatically contribute to the community.<br>
+The system is designed to run without any interaction or maintenance and automatically contributes to the community.<br>
 We know, for some this may not be enough. So here come some ways to further inspect the system and change configuration parameters.
 
 <a name="ssh"></a>
 ## SSH and web access
-By default, the SSH daemon only allows access on **tcp/64295** with a user / password combination from RFC1918 networks. However, if you want to be able to login remotely via SSH you need to put your SSH keys on the host as described below.<br>
+By default, the SSH daemon allows access on **tcp/64295** with a user / password combination and prevents credential brute forcing attempts using `fail2ban`. This also counts for Admin UI (**tcp/64294**) and Web UI (**tcp/64297**) access.<br>
-It is configured to prevent password login from official IP addresses and pubkey-authentication must be used. Copy your SSH keyfile to `/home/tsec/.ssh/authorized_keys` and set the appropriate permissions (`chmod 600 authorized_keys`) as well as the correct ownership (`chown tsec:tsec authorized_keys`).
 
-If you do not have a SSH client at hand and still want to access the machine via SSH you can do so by directing your browser to `https://<your.ip>:64297`, enter
+If you do not have a SSH client at hand and still want to access the machine via command line you can do so by accessing the Admin UI from `https://<your.ip>:64294`, enter
 
-- user: **user you chose during the installation**
+- user: **[tsec or user]** *you chose during one of the post install methods*
-- pass: **password you chose during the installation**
+- pass: **[password]** *you chose during the installation*
 
-and choose **WebTTY** from the navigation bar. You will be prompted to allow access for this connection and enter the password for the user **tsec**.
+![Cockpit Terminal](doc/cockpit3.png)
-
-![WebTTY](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/doc/webssh.png)
 
 <a name="kibana"></a>
 ## Kibana Dashboard
-Just open a web browser and access and connect to `https://<your.ip>:64297`, enter
+Just open a web browser and connect to `https://<your.ip>:64297`, enter
 
-- user: **user you chose during the installation**
+- user: **[user]** *you chose during the installation*
-- pass: **password you chose during the installation**
+- pass: **[password]** *you chose during the installation*
 
 and **Kibana** will automagically load. The Kibana dashboard can be customized to fit your needs. By default, we haven't added any filtering, because the filters depend on your setup. E.g. you might want to filter out your incoming administrative ssh connections and connections to update servers.
 
-![Dashbaord](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/doc/dashboard.png)
+![Dashbaord](doc/kibana.png)
 
 <a name="tools"></a>
 ## Tools
 We included some web based management tools to improve and ease up on your daily tasks.
 
-![ES Head Plugin](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/doc/headplugin.png)
+![Cockpit Overview](doc/cockpit1.png)
-![Netdata](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/doc/netdata.png)
+
-![Portainer](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/doc/dockerui.png)
+![Cockpit Containers](doc/cockpit2.png)
-![Spiderfoot](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/doc/spiderfoot.png)
+
+![Cyberchef](doc/cyberchef.png)
+
+![ES Head Plugin](doc/headplugin.png)
+
+![Spiderfoot](doc/spiderfoot.png)
 
 
 <a name="maintenance"></a>
 ## Maintenance
-As mentioned before, the system was designed to be low maintenance. Basically, there is nothing you have to do but let it run.
+As mentioned before, the system is designed to be low maintenance. Basically, there is nothing you have to do but let it run.
 
 If you run into any problems, a reboot may fix it :bowtie:
 
@@ -363,8 +460,8 @@ If new versions of the components involved appear, we will test them and build n
 
 <a name="submission"></a>
 ## Community Data Submission
-We provide T-Pot in order to make it accessible to all parties interested in honeypot deployment. By default, the data captured is submitted to a community backend. This community backend uses the data to feed a [community data view](http://sicherheitstacho.eu/?peers=communityPeers), a separate channel on our own [Sicherheitstacho](http://sicherheitstacho.eu), which is powered by our own set of honeypots.
+We provide T-Pot in order to make it accessible to all parties interested in honeypot deployment. By default, the captured data is submitted to a community backend. This community backend uses the data to feed [Sicherheitstacho](https://sicherheitstacho.eu).
-You may opt out the submission to our community server by removing the `# Ewsposter service` from `/opt/tpot/etc/tpot.yml`:
+You may opt out of the submission by removing the `# Ewsposter service` from `/opt/tpot/etc/tpot.yml`:
 1. Stop T-Pot services: `systemctl stop tpot`
 2. Remove Ewsposter service: `vi /opt/tpot/etc/tpot.yml`
 3. Remove the following lines, save and exit vi (`:x!`):<br>
@@ -375,7 +472,7 @@ You may opt out the submission to our community server by removing the `# Ewspos
     restart: always
     networks:
      - ewsposter_local
-    image: "dtagdevsec/ewsposter:1710"
+    image: "dtagdevsec/ewsposter:1903"
     volumes:
      - /data:/data
      - /data/ews/conf/ews.ip:/opt/ewsposter/ews.ip
@@ -386,13 +483,24 @@ Data is submitted in a structured ews-format, a XML stucture. Hence, you can par
 
 We encourage you not to disable the data submission as it is the main purpose of the community approach - as you all know **sharing is caring** 😍
 
+<a name="hpfeeds-optin"></a>
+## Opt-In HPFEEDS Data Submission
+As an Opt-In it is now possible to also share T-Pot data with 3rd party HPFEEDS brokers, such as [SISSDEN](https://sissden.eu).  
+If you want to share your T-Pot data you simply have to register an account with a 3rd party broker with its own benefits towards the community. Once registered you will receive your credentials to share events with the broker. In T-Pot you simply run `hpfeeds_optin.sh` which will ask for your credentials, in case of SISSDEN this is just `Ident` and `Secret`, everything else is pre-configured.  
+It will automatically update `/opt/tpot/etc/tpot.yml` to deliver events to your desired broker.
+
+The script can accept a config file as an argument, e.g. `./hpfeeds_optin.sh --conf=hpfeeds.cfg`
+
+Your current config will also be stored in `/data/ews/conf/hpfeeds.cfg` where you can review or change it.  
+Be sure to apply any changes by running `./hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg`.  
+No worries: Your old config gets backed up in `/data/ews/conf/hpfeeds.cfg.old`
+
+Of course you can also rerun the `hpfeeds_optin.sh` script to change and apply your settings interactively.
+
 <a name="roadmap"></a>
 # Roadmap
 As with every development there is always room for improvements ...
 
-- Introduce new honeypots
-- Improve automatic updates
-
 Some features may be provided with updated docker images, others may require some hands on from your side.
 
 You are always invited to participate in development on our [GitHub](https://github.com/dtag-dev-sec/tpotce) page.
@@ -402,7 +510,7 @@ You are always invited to participate in development on our [GitHub](https://git
 - We don't have access to your system. So we cannot remote-assist when you break your configuration. But you can simply reinstall.
 - The software was designed with best effort security, not to be in stealth mode. Because then, we probably would not be able to provide those kind of honeypot services.
 - You install and you run within your responsibility. Choose your deployment wisely as a system compromise can never be ruled out.
-- Honeypots should - by design - not host any sensitive data. Make sure you don't add any.
+- Honeypots should - by design - may not host any sensitive data. Make sure you don't add any.
 - By default, your data is submitted to the community dashboard. You can disable this in the config. But hey, wouldn't it be better to contribute to the community?
 
 <a name="faq"></a>
@@ -414,53 +522,52 @@ Please report any issues or questions on our [GitHub issue list](https://github.
 We provide the software **as is** in a Community Edition format. T-Pot is designed to run out of the box and with zero maintenance involved. <br>
 We hope you understand that we cannot provide support on an individual basis. We will try to address questions, bugs and problems on our [GitHub issue list](https://github.com/dtag-dev-sec/tpotce/issues).
 
-For general feedback you can write to cert @ telekom.de.
-
 <a name="licenses"></a>
 # Licenses
 The software that T-Pot is built on uses the following licenses.
-<br>GPLv2: [conpot (by Lukas Rist)](https://github.com/mushorg/conpot/blob/master/LICENSE.txt), [dionaea](https://github.com/DinoTools/dionaea/blob/master/LICENSE), [honeytrap (by Tillmann Werner)](https://github.com/armedpot/honeytrap/blob/master/LICENSE), [suricata](http://suricata-ids.org/about/open-source/)
+<br>GPLv2: [conpot](https://github.com/mushorg/conpot/blob/master/LICENSE.txt), [dionaea](https://github.com/DinoTools/dionaea/blob/master/LICENSE), [honeypy](https://github.com/foospidy/HoneyPy/blob/master/LICENSE), [honeytrap](https://github.com/armedpot/honeytrap/blob/master/LICENSE), [suricata](http://suricata-ids.org/about/open-source/)
-<br>GPLv3: [elasticpot (by Markus Schmall)](https://github.com/schmalle/ElasticPot), [emobility (by Mohamad Sbeiti)](https://github.com/dtag-dev-sec/emobility/blob/master/LICENSE), [ewsposter (by Markus Schroer)](https://github.com/dtag-dev-sec/ews/), [glastopf (by Lukas Rist)](https://github.com/glastopf/glastopf/blob/master/GPL), [rdpy](https://github.com/citronneur/rdpy/blob/master/LICENSE), [netdata](https://github.com/firehol/netdata/blob/master/LICENSE.md)
+<br>GPLv3: [adbhoney](https://github.com/huuck/ADBHoney), [elasticpot](https://github.com/schmalle/ElasticPot), [ewsposter](https://github.com/dtag-dev-sec/ews/), [fatt](https://github.com/0x4D31/fatt/blob/master/LICENSE), [rdpy](https://github.com/citronneur/rdpy/blob/master/LICENSE), [heralding](https://github.com/johnnykv/heralding/blob/master/LICENSE.txt), [snare](https://github.com/mushorg/snare/blob/master/LICENSE), [tanner](https://github.com/mushorg/snare/blob/master/LICENSE)
-<br>Apache 2 License: [elasticsearch](https://github.com/elasticsearch/elasticsearch/blob/master/LICENSE.txt), [logstash](https://github.com/elasticsearch/logstash/blob/master/LICENSE), [kibana](https://github.com/elasticsearch/kibana/blob/master/LICENSE.md), [docker](https://github.com/docker/docker/blob/master/LICENSE), [elasticsearch-head](https://github.com/mobz/elasticsearch-head/blob/master/LICENCE)
+<br>Apache 2 License: [cyberchef](https://github.com/gchq/CyberChef/blob/master/LICENSE), [elasticsearch](https://github.com/elasticsearch/elasticsearch/blob/master/LICENSE.txt), [logstash](https://github.com/elasticsearch/logstash/blob/master/LICENSE), [kibana](https://github.com/elasticsearch/kibana/blob/master/LICENSE.md), [docker](https://github.com/docker/docker/blob/master/LICENSE), [elasticsearch-head](https://github.com/mobz/elasticsearch-head/blob/master/LICENCE)
-<br>MIT License: [ctop](https://github.com/bcicen/ctop/blob/master/LICENSE), [wetty](https://github.com/krishnasrinivas/wetty/blob/master/LICENSE)
+<br>MIT license: [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot/blob/master/LICENSE), [glutton](https://github.com/mushorg/glutton/blob/master/LICENSE)
-<br>zlib License: [vnclowpot](https://github.com/magisterquis/vnclowpot/blob/master/LICENSE)
+<br> Other: [cowrie](https://github.com/micheloosterhof/cowrie/blob/master/LICENSE.md), [mailoney](https://github.com/awhitehatter/mailoney), [Debian licensing](https://www.debian.org/legal/licenses/)
-<br>[cowrie (copyright disclaimer by Upi Tamminen)](https://github.com/micheloosterhof/cowrie/blob/master/doc/COPYRIGHT)
-<br>[mailoney](https://github.com/awhitehatter/mailoney)
-<br>[Ubuntu licensing](http://www.ubuntu.com/about/about-ubuntu/licensing)
-<br>[Portainer](https://github.com/portainer/portainer/blob/develop/LICENSE)
 
 <a name="credits"></a>
 # Credits
-Without open source and the fruitful development community we are proud to be a part of, T-Pot would not have been possible! Our thanks are extended but not limited to the following people and organizations:
+Without open source and the fruitful development community (we are proud to be a part of), T-Pot would not have been possible! Our thanks are extended but not limited to the following people and organizations:
 
 ### The developers and development communities of
 
+* [adbhoney](https://github.com/huuck/ADBHoney/graphs/contributors)
+* [apt-fast](https://github.com/ilikenwf/apt-fast/graphs/contributors)
+* [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot/graphs/contributors)
+* [cockpit](https://github.com/cockpit-project/cockpit/graphs/contributors)
 * [conpot](https://github.com/mushorg/conpot/graphs/contributors)
 * [cowrie](https://github.com/micheloosterhof/cowrie/graphs/contributors)
+* [debian](http://www.debian.org/)
 * [dionaea](https://github.com/DinoTools/dionaea/graphs/contributors)
 * [docker](https://github.com/docker/docker/graphs/contributors)
-* [elasticpot](https://github.com/schmalle/ElasticPot/graphs/contributors)
+* [elasticpot](https://github.com/schmalle/ElasticpotPY/graphs/contributors)
 * [elasticsearch](https://github.com/elastic/elasticsearch/graphs/contributors)
 * [elasticsearch-head](https://github.com/mobz/elasticsearch-head/graphs/contributors)
-* [emobility](https://github.com/dtag-dev-sec/emobility/graphs/contributors)
 * [ewsposter](https://github.com/armedpot/ewsposter/graphs/contributors)
-* [glastopf](https://github.com/mushorg/glastopf/graphs/contributors)
+* [fatt](https://github.com/0x4D31/fatt/graphs/contributors)
+* [glutton](https://github.com/mushorg/glutton/graphs/contributors)
+* [heralding](https://github.com/johnnykv/heralding/graphs/contributors)
+* [honeypy](https://github.com/foospidy/HoneyPy/graphs/contributors)
 * [honeytrap](https://github.com/armedpot/honeytrap/graphs/contributors)
 * [kibana](https://github.com/elastic/kibana/graphs/contributors)
 * [logstash](https://github.com/elastic/logstash/graphs/contributors)
 * [mailoney](https://github.com/awhitehatter/mailoney)
-* [netdata](https://github.com/firehol/netdata/graphs/contributors)
+* [medpot](https://github.com/schmalle/medpot/graphs/contributors)
 * [p0f](http://lcamtuf.coredump.cx/p0f3/)
-* [portainer](https://github.com/portainer/portainer/graphs/contributors)
 * [rdpy](https://github.com/citronneur/rdpy)
 * [spiderfoot](https://github.com/smicallef/spiderfoot)
+* [snare](https://github.com/mushorg/snare/graphs/contributors)
+* [tanner](https://github.com/mushorg/tanner/graphs/contributors)
 * [suricata](https://github.com/inliniac/suricata/graphs/contributors)
-* [ubuntu](http://www.ubuntu.com/)
-* [vnclowpot](https://github.com/magisterquis/vnclowpot)
-* [wetty](https://github.com/krishnasrinivas/wetty/graphs/contributors)
 
 ### The following companies and organizations
-* [canonical](http://www.canonical.com/)
+* [debian](https://www.debian.org/)
 * [docker](https://www.docker.com/)
 * [elastic.io](https://www.elastic.co/)
 * [honeynet project](https://www.honeynet.org/)
@@ -472,7 +579,12 @@ Without open source and the fruitful development community we are proud to be a
 # Stay tuned ...
 We will be releasing a new version of T-Pot about every 6-12 months.
 
+<a name="testimonial"></a>
+# Testimonial
+One of the greatest feedback we have gotten so far is by one of the Conpot developers:<br>
+***"[...] I highly recommend T-Pot which is ... it's not exactly a swiss army knife .. it's more like a swiss army soldier, equipped with a swiss army knife. Inside a tank. A swiss tank. [...]"***
+
 <a name="funfact"></a>
 # Fun Fact
 
-Coffee just does not cut it anymore which is why we needed a different caffeine source and consumed *242* bottles of [Club Mate](https://de.wikipedia.org/wiki/Club-Mate) during the development of T-Pot 17.10 😇
+In an effort of saving the environment we are now brewing our own Mate Ice Tea and consumed 73 liters so far for the T-Pot 19.03 development 😇

bin/backup_es_folders.sh

@@ -1,11 +1,19 @@
 #!/bin/bash
+# Run as root only.
+myWHOAMI=$(whoami)
+if [ "$myWHOAMI" != "root" ]
+  then
+    echo "Need to run as root ..."
+    exit
+fi
+
 # Backup all ES relevant folders
 # Make sure ES is available
 myES="http://127.0.0.1:64298/"
 myESSTATUS=$(curl -s -XGET ''$myES'_cluster/health' | jq '.' | grep -c green)
 if ! [ "$myESSTATUS" = "1" ]
   then
-    echo "### Elasticsearch is not available, try starting via 'systemctl start elk'."
+    echo "### Elasticsearch is not available, try starting via 'systemctl start tpot'."
     exit
   else
     echo "### Elasticsearch is available, now continuing."
@@ -16,7 +24,7 @@ fi
 myCOUNT=1
 myDATE=$(date +%Y%m%d%H%M)
 myELKPATH="/data/elk/data"
-myKIBANAINDEXNAME=$(curl -s -XGET ''$myES'_cat/indices/' | grep .kibana | awk '{ print $4 }')
+myKIBANAINDEXNAME=$(curl -s -XGET ''$myES'_cat/indices/.kibana' | awk '{ print $4 }')
 myKIBANAINDEXPATH=$myELKPATH/nodes/0/indices/$myKIBANAINDEXNAME
 
 # Let's ensure normal operation on exit or if interrupted ...

bin/change_ews_config.sh

@@ -0,0 +1,89 @@
+#!/bin/bash
+
+echo """
+
+##############################
+# T-POT DTAG Data Submission #
+# Contact:                   #
+# cert@telekom.de            # 
+##############################
+"""
+
+# Got root?
+myWHOAMI=$(whoami)
+if [ "$myWHOAMI" != "root" ]
+  then
+    echo "Need to run as root ..."
+    sudo ./$0
+    exit
+fi
+
+printf "[*] Enter your API UserID: "
+read apiUser
+printf "[*] Enter your API Token: "
+read apiToken
+printf "[*] If you have multiple T-Pots running, give them each a unique NUMBER, e.g. '2' for your second T-Pot installation. Enter unique number for THIS T-Pot: "
+read indexNumber
+if ! [[ "$indexNumber" =~ ^[0-9]+$ ]]
+    then
+        echo "Sorry integers only. You have to start over..."
+        exit 1
+fi
+apiURL="https://community.sicherheitstacho.eu/ews-0.1/alert/postSimpleMessage"
+printf "[*] Currently, your honeypot is configured to transmit data the default backend at 'https://community.sicherheitstacho.eu/ews-0.1/alert/postSimpleMessage'. Do you want to change this API endpoint? Only do this if you run your own PEBA backend instance? (N/y): "
+read replyAPI
+if [[ $replyAPI =~ ^[Yy]$ ]]
+then    
+    printf "[*] Enter your API endpoint URL and make sure it contains the full path, e.g. 'https://myDomain.local:9922/ews-0.1/alert/postSimpleMessage': "
+    read apiURL
+fi
+
+
+
+echo ""
+echo "[*] Recap! You defined: "
+echo "############################"
+echo "API User: " $apiUser
+echo "API Token: " $apiToken
+echo "API URL: " $apiURL
+echo "Unique numeric ID for your T-Pot Installation: "  $indexNumber
+echo "Specific honeypot-IDs will look like : <honeypotType>-"$apiUser"-"$indexNumber
+echo "############################"
+echo ""
+printf  "[*] Is the above correct (y/N)? "
+read reply
+if [[ ! $reply =~ ^[Yy]$ ]]
+then	
+	echo "OK, then run this again..."
+    exit 1
+fi
+echo ""
+echo "[+] Creating config file with API UserID '$apiUser' and API Token '$apiToken'."
+echo "[+] Fetching config file from github. Outgoing https requests must be enabled!"
+wget -q https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/docker/ews/dist/ews.cfg -O ews.cfg.dist 
+if [[ -f "ews.cfg.dist" ]]; then
+	echo "[+] Successfully downloaded ews.cfg from github."
+else 
+	echo "[+] Could not download ews.cfg from github."
+	exit 1
+fi 
+echo "[+] Patching ews.cfg API Credentials."
+sed 's/community-01-user/'$apiUser'/' ews.cfg.dist > ews.cfg
+sed -i 's/foth{a5maiCee8fineu7/'$apiToken'/' ews.cfg
+echo "[+] Patching ews.cfg API Url."
+apiURL=${apiURL////\\/};
+sed -i 's/https:\/\/community.sicherheitstacho.eu\/ews-0.1\/alert\/postSimpleMessage/'$apiURL'/' ews.cfg
+echo "[+] Patching ews.cfg honeypot IDs."
+sed -i 's/community-01/'$apiUser'-'$indexNumber'/' ews.cfg
+
+rm ews.cfg.dist
+
+echo "[+] Changing tpot.yml to include new ews.cfg."
+
+cp ews.cfg /data/ews/conf/ews.cfg
+cp /opt/tpot/etc/tpot.yml /opt/tpot/etc/tpot.yml.bak
+sed -i '/- \/data\/ews\/conf\/ews.ip:\/opt\/ewsposter\/ews.ip/a\ \ \   - \/data\/ews\/conf\/ews.cfg:\/opt\/ewsposter\/ews.cfg' /opt/tpot/etc/tpot.yml
+
+echo "[+] Restarting T-Pot."
+systemctl restart tpot
+echo "[+] Done."

bin/clean.sh

@@ -1,6 +1,5 @@
 #!/bin/bash
 # T-Pot Container Data Cleaner & Log Rotator
-
 # Set colors
 myRED=""
 myGREEN=""
@@ -20,6 +19,8 @@ echo $(ls $myFOLDER | wc -l)
 fuLOGROTATE () {
   local mySTATUS="/opt/tpot/etc/logrotate/status"
   local myCONF="/opt/tpot/etc/logrotate/logrotate.conf"
+  local myADBHONEYTGZ="/data/adbhoney/downloads.tgz"
+  local myADBHONEYDL="/data/adbhoney/downloads/"
   local myCOWRIETTYLOGS="/data/cowrie/log/tty/"
   local myCOWRIETTYTGZ="/data/cowrie/log/ttylogs.tgz"
   local myCOWRIEDL="/data/cowrie/downloads/"
@@ -32,43 +33,65 @@ fuLOGROTATE () {
   local myHONEYTRAPATTACKSTGZ="/data/honeytrap/attacks.tgz"
   local myHONEYTRAPDL="/data/honeytrap/downloads/"
   local myHONEYTRAPDLTGZ="/data/honeytrap/downloads.tgz"
+  local myTANNERF="/data/tanner/files/"
+  local myTANNERFTGZ="/data/tanner/files.tgz"
 
 # Ensure correct permissions and ownerships for logrotate to run without issues
-chmod 760 /data/ -R
+chmod 770 /data/ -R
 chown tpot:tpot /data -R
+chmod 644 /data/nginx/conf -R
+chmod 644 /data/nginx/cert -R
 
 # Run logrotate with force (-f) first, so the status file can be written and race conditions (with tar) be avoided
 logrotate -f -s $mySTATUS $myCONF
 
 # Compressing some folders first and rotate them later
+if [ "$(fuEMPTY $myADBHONEYDL)" != "0" ]; then tar cvfz $myADBHONEYTGZ $myADBHONEYDL; fi
 if [ "$(fuEMPTY $myCOWRIETTYLOGS)" != "0" ]; then tar cvfz $myCOWRIETTYTGZ $myCOWRIETTYLOGS; fi
 if [ "$(fuEMPTY $myCOWRIEDL)" != "0" ]; then tar cvfz $myCOWRIEDLTGZ $myCOWRIEDL; fi
 if [ "$(fuEMPTY $myDIONAEABI)" != "0" ]; then tar cvfz $myDIONAEABITGZ $myDIONAEABI; fi
 if [ "$(fuEMPTY $myDIONAEABIN)" != "0" ]; then tar cvfz $myDIONAEABINTGZ $myDIONAEABIN; fi
 if [ "$(fuEMPTY $myHONEYTRAPATTACKS)" != "0" ]; then tar cvfz $myHONEYTRAPATTACKSTGZ $myHONEYTRAPATTACKS; fi
 if [ "$(fuEMPTY $myHONEYTRAPDL)" != "0" ]; then tar cvfz $myHONEYTRAPDLTGZ $myHONEYTRAPDL; fi
+if [ "$(fuEMPTY $myTANNERF)" != "0" ]; then tar cvfz $myTANNERFTGZ $myTANNERF; fi
 
 # Ensure correct permissions and ownership for previously created archives
-chmod 760 $myCOWRIETTYTGZ $myCOWRIEDLTGZ $myDIONAEABITGZ $myDIONAEABINTGZ $myHONEYTRAPATTACKSTGZ $myHONEYTRAPDLTGZ
+chmod 770 $myADBHONEYTGZ $myCOWRIETTYTGZ $myCOWRIEDLTGZ $myDIONAEABITGZ $myDIONAEABINTGZ $myHONEYTRAPATTACKSTGZ $myHONEYTRAPDLTGZ $myTANNERFTGZ
-chown tpot:tpot $myCOWRIETTYTGZ $myCOWRIEDLTGZ $myDIONAEABITGZ $myDIONAEABINTGZ $myHONEYTRAPATTACKSTGZ $myHONEYTRAPDLTGZ
+chown tpot:tpot $myADBHONEYTGZ $myCOWRIETTYTGZ $myCOWRIEDLTGZ $myDIONAEABITGZ $myDIONAEABINTGZ $myHONEYTRAPATTACKSTGZ $myHONEYTRAPDLTGZ $myTANNERFTGZ
 
 # Need to remove subfolders since too many files cause rm to exit with errors
-rm -rf $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL
+rm -rf $myADBHONEYDL $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL $myTANNERF
 
 # Recreate subfolders with correct permissions and ownership
-mkdir -p $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL
+mkdir -p $myADBHONEYDL $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL $myTANNERF
-chmod 760 $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL 
+chmod 770 $myADBHONEYDL $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL $myTANNERF
-chown tpot:tpot $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL
+chown tpot:tpot $myADBHONEYDL $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL $myTANNERF
 
 # Run logrotate again to account for previously created archives - DO NOT FORCE HERE!
 logrotate -s $mySTATUS $myCONF
 }
 
+# Let's create a function to clean up and prepare honeytrap data
+fuADBHONEY () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/adbhoney/*; fi
+  mkdir -p /data/adbhoney/log/ /data/adbhoney/downloads/
+  chmod 770 /data/adbhoney/ -R
+  chown tpot:tpot /data/adbhoney/ -R
+}
+
+# Let's create a function to clean up and prepare ciscoasa data
+fuCISCOASA () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/ciscoasa/*; fi
+  mkdir -p /data/ciscoasa/log
+  chmod 770 /data/ciscoasa -R
+  chown tpot:tpot /data/ciscoasa -R
+}
+
 # Let's create a function to clean up and prepare conpot data
 fuCONPOT () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/conpot/*; fi
   mkdir -p /data/conpot/log
-  chmod 760 /data/conpot -R
+  chmod 770 /data/conpot -R
   chown tpot:tpot /data/conpot -R
 }
 
@@ -76,7 +99,7 @@ fuCONPOT () {
 fuCOWRIE () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/cowrie/*; fi
   mkdir -p /data/cowrie/log/tty/ /data/cowrie/downloads/ /data/cowrie/keys/ /data/cowrie/misc/
-  chmod 760 /data/cowrie -R
+  chmod 770 /data/cowrie -R
   chown tpot:tpot /data/cowrie -R
 }
 
@@ -84,7 +107,7 @@ fuCOWRIE () {
 fuDIONAEA () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/dionaea/*; fi
   mkdir -p /data/dionaea/log /data/dionaea/bistreams /data/dionaea/binaries /data/dionaea/rtp /data/dionaea/roots/ftp /data/dionaea/roots/tftp /data/dionaea/roots/www /data/dionaea/roots/upnp
-  chmod 760 /data/dionaea -R
+  chmod 770 /data/dionaea -R
   chown tpot:tpot /data/dionaea -R
 }
 
@@ -92,7 +115,7 @@ fuDIONAEA () {
 fuELASTICPOT () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/elasticpot/*; fi
   mkdir -p /data/elasticpot/log
-  chmod 760 /data/elasticpot -R
+  chmod 770 /data/elasticpot -R
   chown tpot:tpot /data/elasticpot -R
 }
 
@@ -102,31 +125,47 @@ fuELK () {
   # ELK daemon log files will be removed
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/elk/log/*; fi
   mkdir -p /data/elk
-  chmod 760 /data/elk -R
+  chmod 770 /data/elk -R
   chown tpot:tpot /data/elk -R
 }
 
-# Let's create a function to clean up and prepare emobility data
+# Let's create a function to clean up and prepare fatt data
-fuEMOBILITY () {
+fuFATT () {
-  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/emobility/*; fi
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/fatt/*; fi
-  mkdir -p /data/emobility/log
+  mkdir -p /data/fatt/log
-  chmod 760 /data/emobility -R
+  chmod 770 -R /data/fatt
-  chown tpot:tpot /data/emobility -R
+  chown tpot:tpot -R /data/fatt
 }
 
 # Let's create a function to clean up and prepare glastopf data
-fuGLASTOPF () {
+fuGLUTTON () {
-  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/glastopf/*; fi
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/glutton/*; fi
-  mkdir -p /data/glastopf
+  mkdir -p /data/glutton/log
-  chmod 760 /data/glastopf -R
+  chmod 770 /data/glutton -R
-  chown tpot:tpot /data/glastopf -R
+  chown tpot:tpot /data/glutton -R
+}
+
+# Let's create a function to clean up and prepare heralding data
+fuHERALDING () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/heralding/*; fi
+  mkdir -p /data/heralding/log
+  chmod 770 /data/heralding -R
+  chown tpot:tpot /data/heralding -R
+}
+
+# Let's create a function to clean up and prepare honeypy data
+fuHONEYPY () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/honeypy/*; fi
+  mkdir -p /data/honeypy/log
+  chmod 770 /data/honeypy -R
+  chown tpot:tpot /data/honeypy -R
 }
 
 # Let's create a function to clean up and prepare honeytrap data
 fuHONEYTRAP () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/honeytrap/*; fi
   mkdir -p /data/honeytrap/log/ /data/honeytrap/attacks/ /data/honeytrap/downloads/
-  chmod 760 /data/honeytrap/ -R
+  chmod 770 /data/honeytrap/ -R
   chown tpot:tpot /data/honeytrap/ -R
 }
 
@@ -134,15 +173,31 @@ fuHONEYTRAP () {
 fuMAILONEY () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/mailoney/*; fi
   mkdir -p /data/mailoney/log/
-  chmod 760 /data/mailoney/ -R
+  chmod 770 /data/mailoney/ -R
   chown tpot:tpot /data/mailoney/ -R
 }
 
+# Let's create a function to clean up and prepare mailoney data
+fuMEDPOT () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/medpot/*; fi
+  mkdir -p /data/medpot/log/
+  chmod 770 /data/medpot/ -R
+  chown tpot:tpot /data/medpot/ -R
+}
+
+# Let's create a function to clean up nginx logs
+fuNGINX () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/nginx/log/*; fi
+  touch /data/nginx/log/error.log
+  chmod 644 /data/nginx/conf -R
+  chmod 644 /data/nginx/cert -R
+}
+
 # Let's create a function to clean up and prepare rdpy data
 fuRDPY () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/rdpy/*; fi
   mkdir -p /data/rdpy/log/
-  chmod 760 /data/rdpy/ -R
+  chmod 770 /data/rdpy/ -R
   chown tpot:tpot /data/rdpy/ -R
 }
 
@@ -150,7 +205,7 @@ fuRDPY () {
 fuSPIDERFOOT () {
   mkdir -p /data/spiderfoot
   touch /data/spiderfoot/spiderfoot.db
-  chmod 760 -R /data/spiderfoot
+  chmod 770 -R /data/spiderfoot
   chown tpot:tpot -R /data/spiderfoot
 }
 
@@ -158,7 +213,7 @@ fuSPIDERFOOT () {
 fuSURICATA () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/suricata/*; fi
   mkdir -p /data/suricata/log
-  chmod 760 -R /data/suricata
+  chmod 770 -R /data/suricata
   chown tpot:tpot -R /data/suricata
 }
 
@@ -166,19 +221,18 @@ fuSURICATA () {
 fuP0F () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/p0f/*; fi
   mkdir -p /data/p0f/log
-  chmod 760 -R /data/p0f
+  chmod 770 -R /data/p0f
   chown tpot:tpot -R /data/p0f
 }
 
-# Let's create a function to clean up and prepare vnclowpot data
+# Let's create a function to clean up and prepare p0f data
-fuVNCLOWPOT () {
+fuTANNER () {
-  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/vnclowpot/*; fi
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/tanner/*; fi
-  mkdir -p /data/vnclowpot/log/
+  mkdir -p /data/tanner/log /data/tanner/files
-  chmod 760 /data/vnclowpot/ -R
+  chmod 770 -R /data/tanner
-  chown tpot:tpot /data/vnclowpot/ -R
+  chown tpot:tpot -R /data/tanner
 }
 
-
 # Avoid unwanted cleaning
 if [ "$myPERSISTENCE" = "" ];
   then
@@ -201,19 +255,24 @@ if [ "$myPERSISTENCE" = "on" ];
     fuLOGROTATE
   else
     echo "Cleaning up and preparing data folders."
+    fuADBHONEY
+    fuCISCOASA
     fuCONPOT
     fuCOWRIE
     fuDIONAEA
     fuELASTICPOT
     fuELK
-    fuEMOBILITY
+    fuFATT
-    fuGLASTOPF
+    fuGLUTTON
+    fuHERALDING
+    fuHONEYPY
     fuHONEYTRAP
     fuMAILONEY
+    fuMEDPOT
+    fuNGINX
     fuRDPY
     fuSPIDERFOOT
     fuSURICATA
     fuP0F
-    fuVNCLOWPOT
+    fuTANNER
   fi
-

bin/dps.sh

@@ -1,65 +1,48 @@
 #/bin/bash
-# Show current status of all running containers
+
+# Run as root only.
+myWHOAMI=$(whoami)
+if [ "$myWHOAMI" != "root" ]
+  then
+    echo "Need to run as root ..."
+    exit
+fi
+
+# Show current status of T-Pot containers
 myPARAM="$1"
-myIMAGES="$(cat /opt/tpot/etc/tpot.yml | grep -v '#' | grep container_name | cut -d: -f2)"
+myCONTAINERS="$(cat /opt/tpot/etc/tpot.yml | grep -v '#' | grep container_name | cut -d: -f2 | sort | tr -d " ")"
 myRED=""
 myGREEN=""
 myBLUE=""
 myWHITE=""
 myMAGENTA=""
 
-function fuCONTAINERSTATUS {
+function fuGETSTATUS {
-local myNAME="$1"
+grc --colour=on docker ps -f status=running -f status=exited --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}" | grep -v "NAME" | sort
-local mySTATUS="$(/usr/bin/docker ps -f name=$myNAME --format "table {{.Status}}" -f status=running -f status=exited | tail -n 1)"
-myDOWN="$(echo "$mySTATUS" | grep -o -E "(STATUS|NAMES|Exited)")"
-
-case "$myDOWN" in
-  STATUS)
-    mySTATUS="$myRED"DOWN"$myWHITE"
-  ;;
-  NAMES)
-    mySTATUS="$myRED"DOWN"$myWHITE"
-  ;;
-  Exited)
-    mySTATUS="$myRED$mySTATUS$myWHITE"
-  ;;
-  *)
-    mySTATUS="$myGREEN$mySTATUS$myWHITE"
-  ;;
-esac
-
-printf "$mySTATUS"
-}
-
-function fuCONTAINERPORTS {
-local myNAME="$1"
-local myPORTS="$(/usr/bin/docker ps -f name=$myNAME --format "table {{.Ports}}" -f status=running -f status=exited | tail -n 1 | sed s/","/",\n\t\t\t\t\t\t\t"/g)"
-
-if [ "$myPORTS" != "PORTS" ];
-  then
-    printf "$myBLUE$myPORTS$myWHITE"
-fi
 }
 
 function fuGETSYS {
 printf "========| System |========\n"
 printf "%+10s %-20s\n" "Date: " "$(date)"
 printf "%+10s %-20s\n" "Uptime: " "$(uptime | cut -b 2-)"
-printf "%+10s %-20s\n" "CPU temp: " "$(sensors | grep 'Physical' | awk '{ print $4" " }' | tr -d [:cntrl:])"
 echo
 }
 
 while true
   do
+    myDPS=$(fuGETSTATUS)
+    myDPSNAMES=$(echo "$myDPS" | awk '{ print $1 }' | sort)
     fuGETSYS
-    printf "%-19s %-36s %s\n" "NAME" "STATUS" "PORTS"
+    printf "%-21s %-28s %s\n" "NAME" "STATUS" "PORTS"
-    for i in $myIMAGES; do
+    if [ "$myDPS" != "" ];
-          myNAME="$myMAGENTA$i$myWHITE"
-          printf "%-32s %-49s %s" "$myNAME" "$(fuCONTAINERSTATUS $i)" "$(fuCONTAINERPORTS $i)"
-          echo
-      if [ "$myPARAM" = "vv" ]; 
       then
-          /usr/bin/docker exec -t "$i" /bin/ps awfuwfxwf | egrep -v -E "awfuwfxwf|/bin/ps" 
+        echo "$myDPS"
+    fi
+    for i in $myCONTAINERS; do
+      myAVAIL=$(echo "$myDPSNAMES" | grep -o "$i" | uniq | wc -l)      	    
+      if [ "$myAVAIL" = "0" ];
+	then
+	  printf "%-28s %-28s\n" "$myRED$i" "DOWN$myWHITE"
       fi
     done
     if [[ $myPARAM =~ ^([1-9]|[1-9][0-9]|[1-9][0-9][0-9])$ ]];

bin/dump_es.sh

@@ -2,10 +2,10 @@
 # Dump all ES data
 # Make sure ES is available
 myES="http://127.0.0.1:64298/"
-myESSTATUS=$(curl -s -XGET ''$myES'_cluster/health' | jq '.' | grep -c green)
+myESSTATUS=$(curl -s -XGET ''$myES'_cluster/health' | jq '.' | grep -c "green\|yellow")
 if ! [ "$myESSTATUS" = "1" ]
   then
-    echo "### Elasticsearch is not available, try starting via 'systemctl start elk'."
+    echo "### Elasticsearch is not available, try starting via 'systemctl start tpot'."
     exit
   else
     echo "### Elasticsearch is available, now continuing."
@@ -20,12 +20,12 @@ trap fuCLEANUP EXIT
 
 # Set vars
 myDATE=$(date +%Y%m%d%H%M)
-myINDICES=$(curl -s -XGET ''$myES'_cat/indices/' | grep logstash | awk '{ print $3 }' | sort | grep -v 1970)
+myINDICES=$(curl -s -XGET ''$myES'_cat/indices/logstash-*' | awk '{ print $3 }' | sort | grep -v 1970)
-myES="http://127.0.0.1:64298/"
+myINDICES+=" .kibana"
 myCOL1=""
 myCOL0=""
 
-# Dumping all ES data
+# Dumping Kibana and Logstash data
 echo $myCOL1"### The following indices will be dumped: "$myCOL0
 echo $myINDICES
 echo
@@ -41,5 +41,5 @@ for i in $myINDICES;
 
 # Build tar archive
 echo $myCOL1"### Now building tar archive: es_dump_"$myDATE".tgz" $myCOL0
-tar cvf es_dump_$myDATE.tar tmp/*
+tar cvf es_dump_$myDATE.tar tmp/.
 echo $myCOL1"### Done."$myCOL0

bin/export_kibana-objects.sh

@@ -1,7 +1,8 @@
 #!/bin/bash
-# Export all Kibana objects
+# Export all Kibana objects through Kibana Saved Objects API
 # Make sure ES is available
 myES="http://127.0.0.1:64298/"
+myKIBANA="http://127.0.0.1:64296/"
 myESSTATUS=$(curl -s -XGET ''$myES'_cluster/health' | jq '.' | grep -c green)
 if ! [ "$myESSTATUS" = "1" ]
   then
@@ -14,10 +15,11 @@ fi
 
 # Set vars
 myDATE=$(date +%Y%m%d%H%M)
-myINDEXCOUNT=$(curl -s -XGET ''$myES'.kibana/index-pattern/logstash-*' | tr '\\' '\n' | grep "scripted" | wc -w)
+myINDEXCOUNT=$(curl -s -XGET ''$myKIBANA'api/saved_objects/_find?type=index-pattern' | jq '.saved_objects[].attributes' | tr '\\' '\n' | grep "scripted" | wc -w)
-myDASHBOARDS=$(curl -s -XGET ''$myES'.kibana/dashboard/_search?filter_path=hits.hits._id&pretty&size=10000'  | jq '.hits.hits[] | {_id}' | jq -r '._id')
+myINDEXID=$(curl -s -XGET ''$myKIBANA'api/saved_objects/_find?type=index-pattern' | jq '.saved_objects[].id' | tr -d '"')
-myVISUALIZATIONS=$(curl -s -XGET ''$myES'.kibana/visualization/_search?filter_path=hits.hits._id&pretty&size=10000' | jq '.hits.hits[] | {_id}' | jq -r '._id')
+myDASHBOARDS=$(curl -s -XGET ''$myKIBANA'api/saved_objects/_find?type=dashboard&per_page=300' | jq '.saved_objects[].id' | tr -d '"')
-mySEARCHES=$(curl -s -XGET ''$myES'.kibana/search/_search?filter_path=hits.hits._id&pretty&size=10000' | jq '.hits.hits[] | {_id}' | jq -r '._id')
+myVISUALIZATIONS=$(curl -s -XGET ''$myKIBANA'api/saved_objects/_find?type=visualization&per_page=300' | jq '.saved_objects[].id' | tr -d '"')
+mySEARCHES=$(curl -s -XGET ''$myKIBANA'api/saved_objects/_find?type=search&per_page=300' | jq '.saved_objects[].id' | tr -d '"')
 myCOL1=""
 myCOL0=""
 
@@ -29,8 +31,8 @@ trap fuCLEANUP EXIT
 
 # Export index patterns
 mkdir -p patterns
-echo $myCOL1"### Now exporting"$myCOL0 $myINDEXCOUNT $myCOL1"index patterns." $myCOL0
+echo $myCOL1"### Now exporting"$myCOL0 $myINDEXCOUNT $myCOL1"index pattern fields." $myCOL0
-curl -s -XGET ''$myES'.kibana/index-pattern/logstash-*?' | jq '._source' > patterns/index-patterns.json
+curl -s -XGET ''$myKIBANA'api/saved_objects/index-pattern/'$myINDEXID'' | jq '. | {attributes}' > patterns/$myINDEXID.json &
 echo
 
 # Export dashboards
@@ -39,7 +41,7 @@ echo $myCOL1"### Now exporting"$myCOL0 $(echo $myDASHBOARDS | wc -w) $myCOL1"das
 for i in $myDASHBOARDS;
   do
     echo $myCOL1"###### "$i $myCOL0
-    curl -s -XGET ''$myES'.kibana/dashboard/'$i'' | jq '._source' > dashboards/$i.json
+    curl -s -XGET ''$myKIBANA'api/saved_objects/dashboard/'$i'' | jq '. | {attributes}' > dashboards/$i.json &
   done;
 echo
 
@@ -49,7 +51,7 @@ echo $myCOL1"### Now exporting"$myCOL0 $(echo $myVISUALIZATIONS | wc -w) $myCOL1
 for i in $myVISUALIZATIONS;
   do
     echo $myCOL1"###### "$i $myCOL0
-    curl -s -XGET ''$myES'.kibana/visualization/'$i'' | jq '._source' > visualizations/$i.json
+    curl -s -XGET ''$myKIBANA'api/saved_objects/visualization/'$i'' | jq '. | {attributes}' > visualizations/$i.json &
   done;
 echo
 
@@ -59,10 +61,13 @@ echo $myCOL1"### Now exporting"$myCOL0 $(echo $mySEARCHES | wc -w) $myCOL1"searc
 for i in $mySEARCHES;
   do
     echo $myCOL1"###### "$i $myCOL0
-    curl -s -XGET ''$myES'.kibana/search/'$i'' | jq '._source' > searches/$i.json
+    curl -s -XGET ''$myKIBANA'api/saved_objects/search/'$i'' | jq '. | {attributes}' > searches/$i.json &
   done;
 echo
 
+# Wait for background exports to finish
+wait
+
 # Building tar archive
 echo $myCOL1"### Now building archive"$myCOL0 "kibana-objects_"$myDATE".tgz"
 tar cvfz kibana-objects_$myDATE.tgz patterns dashboards visualizations searches > /dev/null

bin/hpfeeds_optin.sh

@@ -0,0 +1,153 @@
+#!/bin/bash
+
+# Run as root only.
+myWHOAMI=$(whoami)
+if [ "$myWHOAMI" != "root" ]
+  then
+    echo "Need to run as root ..."
+    exit
+fi
+
+myTPOTYMLFILE="/opt/tpot/etc/tpot.yml"
+
+function fuSISSDEN () {
+echo
+echo "You chose SISSDEN, you just need to provide ident and secret"
+echo
+myENABLE="true"
+myHOST="hpfeeds.sissden.eu"
+myPORT="10000"
+myCHANNEL="t-pot.events"
+myCERT="/opt/ewsposter/sissden.pem"
+read -p "Ident: " myIDENT
+read -p "Secret: " mySECRET
+myFORMAT="json"
+}
+
+function fuGENERIC () {
+echo
+echo "You chose generic, please provide all the details of the broker"
+echo
+myENABLE="true"
+read -p "Host URL: " myHOST
+read -p "Port: " myPORT
+read -p "Channel: " myCHANNEL
+echo "For generic providers set this to 'false'"
+echo "If you received a CA certficate mount it into the ewsposter container by modifying $myTPOTYMLFILE"
+read -p "TLS - 'false' or path to CA in container: " myCERT
+read -p "Ident: " myIDENT
+read -p "Secret: " mySECRET
+read -p "Format ews (xml) or json: " myFORMAT
+}
+
+function fuOPTOUT () {
+echo
+while [ 1 != 2 ]
+  do
+    read -s -n 1 -p "You chose to opt out (y/n)? " mySELECT
+      echo $mySELECT
+      case "$mySELECT" in
+        [y,Y])
+          echo "Opt out."
+          break
+          ;;
+        [n,N])
+          echo "Aborted."
+          exit
+          ;;
+      esac
+done
+myENABLE="false"
+myHOST="host"
+myPORT="port"
+myCHANNEL="channels"
+myCERT="false"
+myIDENT="user"
+mySECRET="secret"
+myFORMAT="json"
+}
+
+function fuWRITETOFILE () {
+if [ -f '/data/ews/conf/hpfeeds.cfg' ]; then
+  echo "Creating backup of current config in /data/ews/conf/hpfeeds.cfg.old"
+  mv /data/ews/conf/hpfeeds.cfg /data/ews/conf/hpfeeds.cfg.old
+fi
+echo "Storing new config in /data/ews/conf/hpfeeds.cfg"
+cat >> /data/ews/conf/hpfeeds.cfg <<EOF
+myENABLE=$myENABLE
+myHOST=$myHOST
+myPORT=$myPORT
+myCHANNEL=$myCHANNEL
+myIDENT=$myIDENT
+mySECRET=$mySECRET
+myCERT=$myCERT
+myFORMAT=$myFORMAT
+EOF
+}
+
+function fuAPPLY () {
+echo "Now stopping T-Pot ..."
+systemctl stop tpot
+echo "Applying your settings to tpot.yml ... "
+sed --follow-symlinks -i "s/EWS_HPFEEDS_ENABLE.*/EWS_HPFEEDS_ENABLE=${myENABLE}/g" "$myTPOTYMLFILE"
+sed --follow-symlinks -i "s/EWS_HPFEEDS_HOST.*/EWS_HPFEEDS_HOST=${myHOST}/g" "$myTPOTYMLFILE"
+sed --follow-symlinks -i "s/EWS_HPFEEDS_PORT.*/EWS_HPFEEDS_PORT=${myPORT}/g" "$myTPOTYMLFILE"
+sed --follow-symlinks -i "s/EWS_HPFEEDS_CHANNELS.*/EWS_HPFEEDS_CHANNELS=${myCHANNEL}/g" "$myTPOTYMLFILE"
+sed --follow-symlinks -i "s#EWS_HPFEEDS_TLSCERT.*#EWS_HPFEEDS_TLSCERT=${myCERT}#g" "$myTPOTYMLFILE"
+sed --follow-symlinks -i "s/EWS_HPFEEDS_IDENT.*/EWS_HPFEEDS_IDENT=${myIDENT}/g" "$myTPOTYMLFILE"
+sed --follow-symlinks -i "s/EWS_HPFEEDS_SECRET.*/EWS_HPFEEDS_SECRET=${mySECRET}/g" "$myTPOTYMLFILE"
+sed --follow-symlinks -i "s/EWS_HPFEEDS_FORMAT.*/EWS_HPFEEDS_FORMAT=${myFORMAT}/g" "$myTPOTYMLFILE"
+echo "Now starting T-Pot ..."
+systemctl start tpot
+echo "You can always change or review your settings in /data/ews/conf/hpfeeds.cfg and apply changes by"
+echo "running \"./hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg\""
+echo "Done."
+}
+
+# Check for cmdline argument and parse config file
+filename=$(echo $@ | cut -d= -f2)
+if [ $# == 1 ] && echo $@ | grep '\-\-conf=' > /dev/null && [ ! -z $filename ] && [ -f $filename ]
+  then
+    source $filename
+else
+
+# Proceed with interactive setup when no config file is found
+echo "HPFEEDS Delivery Opt-In for T-Pot"
+echo "---------------------------------"
+echo "By running this script you agree to share your data with a 3rd party and agree to their corresponding sharing terms."
+echo
+echo
+echo "Please choose your broker"
+echo "---------------------------"
+echo "[1] - SISSDEN"
+echo "[2] - Generic (enter details manually)"
+echo "[0] - Opt out of HPFEEDS"
+echo "[q] - Do not agree end exit"
+echo
+while [ 1 != 2 ]
+  do
+    read -s -n 1 -p "Your choice: " mySELECT
+      echo $mySELECT
+      case "$mySELECT" in
+        [1])
+	  fuSISSDEN
+          break
+          ;;
+        [2])
+	  fuGENERIC
+          break
+          ;;
+        [0])
+	  fuOPTOUT
+          break
+          ;;
+        [q,Q])
+	  echo "Aborted."
+          exit
+          ;;
+      esac
+done
+
+fi
+fuWRITETOFILE
+fuAPPLY

bin/hptest.sh

@@ -0,0 +1,118 @@
+#!/bin/bash
+
+myHOST="$1"
+myPACKAGES="netcat nmap"
+myMEDPOTPACKET="
+MSH|^~\&|ADT1|MCM|LABADT|MCM|198808181126|SECURITY|ADT^A01|MSG00001-|P|2.6
+EVN|A01|198808181123
+PID|||PATID1234^5^M11^^AN||JONES^WILLIAM^A^III||19610615|M||2106-3|677 DELAWARE AVENUE^^EVERETT^MA^02149|GL|(919)379-1212|(919)271-3434~(919)277-3114||S||PATID12345001^2^M10^^ACSN|123456789|9-87654^NC
+NK1|1|JONES^BARBARA^K|SPO|||||20011105
+NK1|1|JONES^MICHAEL^A|FTH
+PV1|1|I|2000^2012^01||||004777^LEBAUER^SIDNEY^J.|||SUR||-||ADM|A0
+AL1|1||^PENICILLIN||CODE16~CODE17~CODE18
+AL1|2||^CAT DANDER||CODE257
+DG1|001|I9|1550|MAL NEO LIVER, PRIMARY|19880501103005|F
+PR1|2234|M11|111^CODE151|COMMON PROCEDURES|198809081123
+ROL|45^RECORDER^ROLE MASTER LIST|AD|RO|KATE^SMITH^ELLEN|199505011201
+GT1|1122|1519|BILL^GATES^A
+IN1|001|A357|1234|BCMD|||||132987
+IN2|ID1551001|SSN12345678
+ROL|45^RECORDER^ROLE MASTER LIST|AD|RO|KATE^ELLEN|199505011201"
+
+function fuGOTROOT {
+myWHOAMI=$(whoami)
+if [ "$myWHOAMI" != "root" ]
+  then
+    echo "Need to run as root ..."
+    exit
+fi
+}
+
+function fuCHECKDEPS {
+myINST=""
+for myDEPS in $myPACKAGES;
+do
+  myOK=$(dpkg -s $myDEPS | grep ok | awk '{ print $3 }');
+  if [ "$myOK" != "ok" ]
+    then
+      myINST=$(echo $myINST $myDEPS)
+  fi
+done
+if [ "$myINST" != "" ]
+  then
+    apt-get update -y
+    for myDEPS in $myINST;
+    do
+      apt-get install $myDEPS -y
+    done
+fi
+}
+
+function fuCHECKFORARGS {
+if [ "$myHOST" != "" ];
+  then
+    echo "All arguments met. Continuing."
+  else
+    echo "Usage: hp_test.sh <[host or ip]>"
+    exit
+fi
+}
+
+function fuGETPORTS {
+myDOCKERCOMPOSEPORTS=$(cat $myDOCKERCOMPOSEYML | yq -r '.services[].ports' | grep ':' | sed -e s/127.0.0.1// | tr -d '", ' | sed -e s/^:// | cut -f1 -d ':' | grep -v "6429\|6430" | sort -gu)
+myPORTS=$(for i in $myDOCKERCOMPOSEPORTS; do echo "$i"; done)
+echo "Found these ports enabled:"
+echo "$myPORTS"
+exit
+}
+
+function fuSCAN {
+local myTIMEOUT="$1"
+local mySCANPORT="$2"
+local mySCANIP="$3"
+local mySCANOPTS="$4"
+
+timeout --foreground ${myTIMEOUT} nmap ${mySCANOPTS} -T4 -v -p ${mySCANPORT} ${mySCANIP} &
+}
+
+# Main
+fuGOTROOT
+fuCHECKDEPS
+fuCHECKFORARGS
+
+echo "Starting scans ..."
+echo "$myMEDPOTPACKET" | nc "$myHOST" 2575 &
+curl -XGET "http://$myHOST:9200/logstash-*/_search" &
+echo "
I20100" | timeout --foreground 3 nc "$myHOST" 10001 &
+fuSCAN "180" "7,8,102,135,161,1025,1080,5000,9200" "$myHOST" "-sC -sS -sU -sV"
+fuSCAN "180" "2048,4096,5432" "$myHOST" "-sC -sS -sU -sV --version-light"
+fuSCAN "120" "20,21" "$myHOST" "--script=ftp* -sC -sS -sV"
+fuSCAN "120" "22" "$myHOST" "--script=ssh2-enum-algos,ssh-auth-methods,ssh-hostkey,ssh-publickey-acceptance,sshv1 -sC -sS -sV"
+fuSCAN "30" "22" "$myHOST" "--script=ssh-brute"
+fuSCAN "120" "23,2323,2324" "$myHOST" "--script=telnet-encryption,telnet-ntlm-info -sC -sS -sV --version-light"
+fuSCAN "120" "25" "$myHOST" "--script=smtp* -sC -sS -sV"
+fuSCAN "180" "42" "$myHOST" "-sC -sS -sV"
+fuSCAN "120" "69" "$myHOST" "--script=tftp-enum -sU"
+fuSCAN "120" "80,81,8080,8443" "$myHOST" "-sC -sS -sV"
+fuSCAN "120" "110,995" "$myHOST" "--script=pop3-capabilities,pop3-ntlm-info -sC -sS -sV --version-light"
+fuSCAN "30" "110,995" "$myHOST" "--script=pop3-brute -sS"
+fuSCAN "120" "143,993" "$myHOST" "--script=imap-capabilities,imap-ntlm-info -sC -sS -sV --version-light"
+fuSCAN "30" "143,993" "$myHOST" "--script=imap-brute -sS"
+fuSCAN "240" "445" "$myHOST" "--script=smb-vuln* -sS -sU"
+fuSCAN "120" "502" "$myHOST" "--script=modbus-discover -sS -sU"
+fuSCAN "120" "623" "$myHOST" "--script=ipmi-cipher-zero,ipmi-version,supermicro-ipmi -sS -sU"
+fuSCAN "30" "623" "$myHOST" "--script=ipmi-brute -sS -sU"
+fuSCAN "120" "1433" "$myHOST" "--script=ms-sql* -sS"
+fuSCAN "120" "1723" "$myHOST" "--script=pptp-version -sS"
+fuSCAN "120" "1883" "$myHOST" "--script=mqtt-subscribe -sS"
+fuSCAN "120" "2404" "$myHOST" "--script=iec-identify -sS"
+fuSCAN "120" "3306" "$myHOST" "--script=mysql-vuln* -sC -sS -sV"
+fuSCAN "120" "3389" "$myHOST" "--script=rdp* -sC -sS -sV"
+fuSCAN "120" "5000" "$myHOST" "--script=*upnp* -sS -sU"
+fuSCAN "120" "5060,5061" "$myHOST" "--script=sip-call-spoof,sip-enum-users,sip-methods -sS -sU"
+fuSCAN "120" "5900" "$myHOST" "--script=vnc-info,vnc-title,realvnc-auth-bypass -sS"
+fuSCAN "120" "27017" "$myHOST" "--script=mongo* -sS"
+fuSCAN "120" "47808" "$myHOST" "--script=bacnet* -sS"
+wait
+reset
+echo "Done."

bin/import_kibana-objects.sh

@@ -2,6 +2,7 @@
 # Import Kibana objects
 # Make sure ES is available
 myES="http://127.0.0.1:64298/"
+myKIBANA="http://127.0.0.1:64296/"
 myESSTATUS=$(curl -s -XGET ''$myES'_cluster/health' | jq '.' | grep -c green)
 if ! [ "$myESSTATUS" = "1" ]
   then
@@ -27,7 +28,7 @@ trap fuCLEANUP EXIT
 if [ "$myDUMP" = "" ];
   then
     echo $myCOL1"### Please provide a backup file name."$myCOL0 
-    echo $myCOL1"### restore-kibana-objects.sh <kibana-objects.tgz>"$myCOL0
+    echo $myCOL1"### import_kibana-objects.sh <kibana-objects.tgz>"$myCOL0
     echo 
     exit
 fi
@@ -41,10 +42,12 @@ fi
 tar xvfz $myDUMP > /dev/null
 
 # Restore index patterns
-myINDEXCOUNT=$(cat patterns/index-patterns.json | tr '\\' '\n' | grep "scripted" | wc -w)
+myINDEXID=$(ls patterns/*.json | cut -c 10- | rev | cut -c 6- | rev)
-echo $myCOL1"### Now importing"$myCOL0 $myINDEXCOUNT $myCOL1"index patterns." $myCOL0
+myINDEXCOUNT=$(cat patterns/$myINDEXID.json | tr '\\' '\n' | grep "scripted" | wc -w)
-curl -s -XDELETE ''$myES'.kibana/index-pattern/logstash-*' > /dev/null
+echo $myCOL1"### Now importing"$myCOL0 $myINDEXCOUNT $myCOL1"index pattern fields." $myCOL0
-curl -s -XPUT ''$myES'.kibana/index-pattern/logstash-*' -T patterns/index-patterns.json > /dev/null
+curl -s -XDELETE ''$myKIBANA'api/saved_objects/index-pattern/logstash-*' -H "Content-Type: application/json" -H "kbn-xsrf: true" > /dev/null
+curl -s -XDELETE ''$myKIBANA'api/saved_objects/index-pattern/'$myINDEXID'' -H "Content-Type: application/json" -H "kbn-xsrf: true" > /dev/null
+curl -s -XPOST ''$myKIBANA'api/saved_objects/index-pattern/'$myINDEXID'' -H "Content-Type: application/json" -H "kbn-xsrf: true" -d @patterns/$myINDEXID.json > /dev/null &
 echo
 
 # Restore dashboards
@@ -52,10 +55,15 @@ myDASHBOARDS=$(ls dashboards/*.json | cut -c 12- | rev | cut -c 6- | rev)
 echo $myCOL1"### Now importing "$myCOL0$(echo $myDASHBOARDS | wc -w)$myCOL1 "dashboards." $myCOL0
 for i in $myDASHBOARDS;
   do
-    echo $myCOL1"###### "$i $myCOL0
+    curl -s -XDELETE ''$myKIBANA'api/saved_objects/dashboard/'$i'' -H "Content-Type: application/json" -H "kbn-xsrf: true" > /dev/null &
-    curl -s -XDELETE ''$myES'.kibana/dashboard/'$i'' > /dev/null
-    curl -s -XPUT ''$myES'.kibana/dashboard/'$i'' -T dashboards/$i.json > /dev/null
   done;
+wait
+for i in $myDASHBOARDS;
+  do
+    echo $myCOL1"###### "$i $myCOL0
+    curl -s -XPOST ''$myKIBANA'api/saved_objects/dashboard/'$i'' -H "Content-Type: application/json" -H "kbn-xsrf: true" -d @dashboards/$i.json > /dev/null &
+  done;
+wait
 echo
 
 # Restore visualizations
@@ -63,22 +71,32 @@ myVISUALIZATIONS=$(ls visualizations/*.json | cut -c 16- | rev | cut -c 6- | rev
 echo $myCOL1"### Now importing "$myCOL0$(echo $myVISUALIZATIONS | wc -w)$myCOL1 "visualizations." $myCOL0
 for i in $myVISUALIZATIONS;
   do
-    echo $myCOL1"###### "$i $myCOL0
+    curl -s -XDELETE ''$myKIBANA'api/saved_objects/visualization/'$i'' -H "Content-Type: application/json" -H "kbn-xsrf: true" > /dev/null &
-    curl -s -XDELETE ''$myES'.kibana/visualization/'$i'' > /dev/null
-    curl -s -XPUT ''$myES'.kibana/visualization/'$i'' -T visualizations/$i.json > /dev/null
   done;
+wait
+for i in $myVISUALIZATIONS;
+  do
+    echo $myCOL1"###### "$i $myCOL0
+    curl -s -XPOST ''$myKIBANA'api/saved_objects/visualization/'$i'' -H "Content-Type: application/json" -H "kbn-xsrf: true" -d @visualizations/$i.json > /dev/null &
+  done;
+wait
 echo
 
 # Restore searches
 mySEARCHES=$(ls searches/*.json | cut -c 10- | rev | cut -c 6- | rev) 
 echo $myCOL1"### Now importing "$myCOL0$(echo $mySEARCHES | wc -w)$myCOL1 "searches." $myCOL0
+for i in $mySEARCHES;
+  do
+    curl -s -XDELETE ''$myKIBANA'api/saved_objects/search/'$i'' -H "Content-Type: application/json" -H "kbn-xsrf: true" > /dev/null &
+  done;
+wait
 for i in $mySEARCHES;
   do
     echo $myCOL1"###### "$i $myCOL0
-    curl -s -XDELETE ''$myES'.kibana/search/'$i'' > /dev/null
+    curl -s -XPOST ''$myKIBANA'api/saved_objects/search/'$i'' -H "Content-Type: application/json" -H "kbn-xsrf: true" -d @searches/$i.json > /dev/null &
-    curl -s -XPUT ''$myES'.kibana/search/'$i'' -T searches/$i.json > /dev/null
   done;
 echo
+wait
 
 # Stats
 echo

bin/restore_es.sh

@@ -2,10 +2,10 @@
 # Restore folder based ES backup
 # Make sure ES is available
 myES="http://127.0.0.1:64298/"
-myESSTATUS=$(curl -s -XGET ''$myES'_cluster/health' | jq '.' | grep -c green)
+myESSTATUS=$(curl -s -XGET ''$myES'_cluster/health' | jq '.' | grep -c "green\|yellow")
 if ! [ "$myESSTATUS" = "1" ]
   then
-    echo "### Elasticsearch is not available, try starting via 'systemctl start elk'."
+    echo "### Elasticsearch is not available, try starting via 'systemctl start tpot'."
     exit
   else
     echo "### Elasticsearch is available, now continuing."
@@ -41,16 +41,50 @@ echo $myCOL1"### Now unpacking tar archive: "$myDUMP $myCOL0
 tar xvf $myDUMP
 
 # Build indices list
-myINDICES=$(ls tmp/logstash*.gz | cut -c 5- | rev | cut -c 4- | rev)
+myINDICES="$(ls tmp/logstash*.gz | cut -c 5- | rev | cut -c 4- | rev)"
+myINDICES+=" .kibana"
 echo $myCOL1"### The following indices will be restored: "$myCOL0
 echo $myINDICES
 echo
 
+# Force single seat template for everything
+echo -n $myCOL1"### Forcing single seat template: "$myCOL0
+curl -s XPUT ''$myES'_template/.*' -H 'Content-Type: application/json' -d'
+{ "index_patterns": ".*",
+  "order": 1,
+  "settings": 
+    { 
+      "number_of_shards": 1,
+      "number_of_replicas": 0 
+    }
+}'
+echo
+
+# Set logstash template
+echo -n $myCOL1"### Setting up logstash template: "$myCOL0
+curl -s XPUT ''$myES'_template/logstash' -H 'Content-Type: application/json' -d'
+{
+  "index_patterns": "logstash-*",
+    "settings" : {
+      "index" : { 
+        "number_of_shards": 1,
+        "number_of_replicas": 0,
+          "mapping" : { 
+            "total_fields" : {
+              "limit" : "2000"
+            } 
+          }  
+      }
+    }
+}'
+echo
+
 # Restore indices
+curl -s -X DELETE ''$myES'.kibana*' > /dev/null
 for i in $myINDICES;
   do
     # Delete index if it already exists
-    curl -s -XDELETE $myES$i > /dev/null
+    curl -s -X DELETE $myES$i > /dev/null
     echo $myCOL1"### Now uncompressing: tmp/$i.gz" $myCOL0
     gunzip -f tmp/$i.gz
     # Restore index to ES

bin/rules.sh

@@ -0,0 +1,107 @@
+#!/bin/bash
+
+### Vars, Ports for Standard services
+myHOSTPORTS="7634 64294 64295"
+myDOCKERCOMPOSEYML="$1"
+myRULESFUNCTION="$2"
+
+function fuCHECKFORARGS {
+### Check if args are present, if not throw error
+
+if [ "$myDOCKERCOMPOSEYML" != "" ] && ([ "$myRULESFUNCTION" == "set" ] || [ "$myRULESFUNCTION" == "unset" ]);
+  then
+    echo "All arguments met. Continuing."
+  else
+    echo "Usage: rules.sh <docker-compose.yml> <[set, unset]>"
+    exit
+fi
+}
+
+function fuNFQCHECK {
+### Check if honeytrap or glutton is actively enabled in docker-compose.yml
+
+myNFQCHECK=$(grep -e '^\s*honeytrap:\|^\s*glutton:' $myDOCKERCOMPOSEYML | tr -d ': ' | uniq)
+if [ "$myNFQCHECK" == "" ];
+  then
+    echo "No NFQ related honeypot detected, no iptables-legacy rules needed. Exiting."
+    exit
+  else
+    echo "Detected $myNFQCHECK as NFQ based honeypot, iptables-legacy rules needed. Continuing."
+fi
+}
+
+function fuGETPORTS {
+### Get ports from docker-compose.yml
+
+myDOCKERCOMPOSEPORTS=$(cat $myDOCKERCOMPOSEYML | yq -r '.services[].ports' | grep ':' | sed -e s/127.0.0.1// | tr -d '", ' | sed -e s/^:// | cut -f1 -d ':' )
+myDOCKERCOMPOSEPORTS+=" $myHOSTPORTS"
+myRULESPORTS=$(for i in $myDOCKERCOMPOSEPORTS; do echo $i; done | sort -gu)
+echo "Setting up / removing these ports:"
+echo "$myRULESPORTS"
+}
+
+function fuSETRULES {
+### Setting up iptables-legacy rules for honeytrap
+if [ "$myNFQCHECK" == "honeytrap" ];
+  then
+    /usr/sbin/iptables-legacy -w -A INPUT -s 127.0.0.1 -j ACCEPT
+    /usr/sbin/iptables-legacy -w -A INPUT -d 127.0.0.1 -j ACCEPT
+
+    for myPORT in $myRULESPORTS; do
+      /usr/sbin/iptables-legacy -w -A INPUT -p tcp --dport $myPORT -j ACCEPT
+    done
+
+    /usr/sbin/iptables-legacy -w -A INPUT -p tcp --syn -m state --state NEW -j NFQUEUE
+fi
+
+### Setting up iptables-legacy rules for glutton
+if [ "$myNFQCHECK" == "glutton" ];
+  then
+    /usr/sbin/iptables-legacy -w -t raw -A PREROUTING -s 127.0.0.1 -j ACCEPT
+    /usr/sbin/iptables-legacy -w -t raw -A PREROUTING -d 127.0.0.1 -j ACCEPT
+
+    for myPORT in $myRULESPORTS; do
+      /usr/sbin/iptables-legacy -w -t raw -A PREROUTING -p tcp --dport $myPORT -j ACCEPT
+    done
+    # No need for NFQ forwarding, such rules are set up by glutton
+fi
+}
+
+function fuUNSETRULES {
+### Removing  iptables-legacy rules for honeytrap
+if [ "$myNFQCHECK" == "honeytrap" ];
+  then
+    /usr/sbin/iptables-legacy -w -D INPUT -s 127.0.0.1 -j ACCEPT
+    /usr/sbin/iptables-legacy -w -D INPUT -d 127.0.0.1 -j ACCEPT
+
+    for myPORT in $myRULESPORTS; do
+      /usr/sbin/iptables-legacy -w -D INPUT -p tcp --dport $myPORT -j ACCEPT
+    done
+
+    /usr/sbin/iptables-legacy -w -D INPUT -p tcp --syn -m state --state NEW -j NFQUEUE
+fi
+
+### Removing iptables-legacy rules for glutton
+if [ "$myNFQCHECK" == "glutton" ];
+  then
+    /usr/sbin/iptables-legacy -w -t raw -D PREROUTING -s 127.0.0.1 -j ACCEPT
+    /usr/sbin/iptables-legacy -w -t raw -D PREROUTING -d 127.0.0.1 -j ACCEPT
+
+    for myPORT in $myRULESPORTS; do
+      /usr/sbin/iptables-legacy -w -t raw -D PREROUTING -p tcp --dport $myPORT -j ACCEPT
+    done
+    # No need for removing NFQ forwarding, such rules are removed by glutton
+fi
+}
+
+# Main
+fuCHECKFORARGS
+fuNFQCHECK
+fuGETPORTS
+
+if [ "$myRULESFUNCTION" == "set" ];
+  then
+    fuSETRULES
+  else
+    fuUNSETRULES
+fi

bin/tped.sh

@@ -0,0 +1,56 @@
+#!/bin/bash
+
+# Run as root only.
+myWHOAMI=$(whoami)
+if [ "$myWHOAMI" != "root" ]
+  then
+    echo "Need to run as root ..."
+    exit
+fi
+
+# set backtitle, get filename
+myBACKTITLE="T-Pot Edition Selection Tool"
+myYMLS=$(cd /opt/tpot/etc/compose/ && ls -1 *.yml)
+myLINK="/opt/tpot/etc/tpot.yml"
+
+# Let's load docker images in parallel
+function fuPULLIMAGES {
+local myTPOTCOMPOSE="/opt/tpot/etc/tpot.yml"
+for name in $(cat $myTPOTCOMPOSE | grep -v '#' | grep image | cut -d'"' -f2 | uniq)
+  do
+    docker pull $name &
+  done
+wait
+echo
+}
+
+# setup menu
+for i in $myYMLS;
+  do
+    myITEMS+="$i $(echo $i | cut -d "." -f1 | tr [:lower:] [:upper:]) " 
+done
+myEDITION=$(dialog --backtitle "$myBACKTITLE" --menu "Select T-Pot Edition" 12 50 5 $myITEMS 3>&1 1>&2 2>&3 3>&-)
+if [ "$myEDITION" == "" ];
+  then
+    echo "Have a nice day!"
+    exit
+fi
+dialog --backtitle "$myBACKTITLE" --title "[ Activate now? ]" --yesno "\n$myEDITION" 7 50
+myOK=$?
+if [ "$myOK" == "0" ];
+  then
+    echo "OK - Activating and downloading latest images."
+    systemctl stop tpot
+    if [ "$(docker ps -aq)" != "" ];
+      then
+        docker stop $(docker ps -aq)
+        docker rm $(docker ps -aq)
+    fi
+    rm -f $myLINK
+    ln -s /opt/tpot/etc/compose/$myEDITION $myLINK
+    fuPULLIMAGES
+    systemctl start tpot
+    echo "Done. Use \"dps.sh\" for monitoring"
+  else 
+    echo "Have a nice day!"
+fi

bin/updateip.sh

@@ -8,9 +8,19 @@ if [ "$myEXTIP" = "" ];
   then
     myEXTIP=$myLOCALIP
 fi
-sed -i "s#IP:.*#IP: $myLOCALIP ($myEXTIP)#" /etc/issue
+mySSHUSER=$(cat /etc/passwd | grep 1000 | cut -d ':' -f1)
-sed -i "s#SSH:.*#SSH: ssh -l tsec -p 64295 $myLOCALIP#" /etc/issue
+echo "" > /etc/issue
-sed -i "s#WEB:.*#WEB: https://$myLOCALIP:64297#" /etc/issue
+toilet -f ivrit -F metal --filter border:metal "T-Pot   19.03" | sed 's/\\/\\\\/g' >> /etc/issue
+echo >> /etc/issue
+echo ",---- [ \n ] [ \d ] [ \t ]" >> /etc/issue
+echo "|" >> /etc/issue
+echo "| IP: $myLOCALIP ($myEXTIP)" >> /etc/issue
+echo "| SSH: ssh -l tsec -p 64295 $myLOCALIP" >> /etc/issue 
+echo "| WEB: https://$myLOCALIP:64297" >> /etc/issue
+echo "| ADMIN: https://$myLOCALIP:64294" >> /etc/issue
+echo "|" >> /etc/issue
+echo "\`----" >> /etc/issue
+echo >> /etc/issue
 tee /data/ews/conf/ews.ip << EOF
 [MAIN]
 ip = $myEXTIP
@@ -21,4 +31,4 @@ MY_INTIP=$myLOCALIP
 MY_HOSTNAME=$HOSTNAME
 EOF
 chown tpot:tpot /data/ews/conf/ews.ip
-chmod 760 /data/ews/conf/ews.ip
+chmod 770 /data/ews/conf/ews.ip

cloud/ansible/.gitignore

@@ -0,0 +1,2 @@
+# Ansible
+*.retry

cloud/ansible/README.md

@@ -0,0 +1,253 @@
+# T-Pot Ansible
+
+Here you can find a ready-to-use solution for your automated T-Pot deployment using [Ansible](https://www.ansible.com/).  
+It consists of an Ansible Playbook with multiple roles, which is reusable for all [OpenStack](https://www.openstack.org/) based clouds (e.g. Open Telekom Cloud, Orange Cloud, Telefonica Open Cloud, OVH) out of the box.  
+Apart from that you can easily adapt the deploy role to use other [cloud providers](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html) (e.g. AWS, Azure, Digital Ocean, Google).
+
+The Playbook first creates a new server and then installs and configures T-Pot.
+
+This example showcases the deployment on our own OpenStack based Public Cloud Offering [Open Telekom Cloud](https://open-telekom-cloud.com/en).
+
+# Table of contents
+- [Preparation of Ansible Master](#ansible-master)
+  - [Ansible Installation](#ansible)
+  - [Agent Forwarding](#agent-forwarding)
+- [Preparations in Open Telekom Cloud Console](#preparation)
+  - [Create new project](#project)
+  - [Create API user](#api-user)
+  - [Import Key Pair](#key-pair)
+  - [Create VPC, Subnet and Security Group](#vpc-subnet-securitygroup)
+- [Clone Git Repository](#clone-git)
+- [Settings and recommended values](#settings)
+  - [OpenStack authentication variables](#os-auth)
+  - [Ansible remote user](#remote-user)
+  - [Instance settings](#instance-settings)
+  - [User password](#user-password)
+  - [Configure `tpot.conf.dist`](#tpot-conf)
+  - [Optional: Custom `ews.cfg`](#ews-cfg)
+  - [Optional: Custom HPFEEDS](#hpfeeds)
+- [Deploying a T-Pot](#deploy)
+- [Further documentation](#documentation)
+
+<a name="ansible-master"></a>
+# Preparation of Ansible Master
+You can either run the Ansible Playbook locally on your Linux or macOS machine or you can use an ECS (Elastic Cloud Server) on Open Telekom Cloud, which I did.  
+I used Ubuntu 18.04 for my Ansible Master Server, but other OSes are fine too.  
+Ansible works over the SSH Port, so you don't have to add any special rules to your Security Group.
+
+<a name="ansible"></a>
+## Ansible Installation
+Example for Ubuntu 18.04:  
+At first we need to add the repository and install Ansible:  
+`sudo apt-add-repository --yes --update ppa:ansible/ansible`  
+`sudo apt install ansible`
+
+For other OSes and Distros have a look at the official [Ansible Documentation](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html).
+
+<a name="agent-forwarding"></a>
+## Agent Forwarding
+Agent Forwarding must be enabled in order to let Ansible do its work.  
+- On Linux or macOS:  
+  - Create or edit `~/.ssh/config`
+  - If you run the Ansible Playbook remotely on your Ansible Master Server:
+    ```
+    Host ANSIBLE_MASTER_IP
+    ForwardAgent yes
+    ```
+  - If you run the Ansible Playbook locally, enable it for all hosts, as this includes newly generated T-Pots:
+    ```
+    Host *
+    ForwardAgent yes
+    ```
+- On Windows using Putty for connecting to your Ansible Master Server:  
+![Putty Agent Forwarding](doc/putty_agent_forwarding.png)
+
+<a name="preparation"></a>
+# Preparations in Open Telekom Cloud Console
+(You can skip this if you have already set up an API account, VPC, Subnet and Security Group)  
+(Just make sure you know the naming for everything, as you will need it to configure the Ansible variables.)
+
+Before we can start deploying, we have to prepare the Open Telekom Cloud tenant.  
+For that, go to the [Web Console](https://auth.otc.t-systems.com/authui/login) and log in with an admin user.
+
+<a name="project"></a>
+## Create new project
+I strongly advise you to create a separate project for the T-Pots in your tenant.  
+In my case I named it `tpot`.
+
+![Create new project](doc/otc_1_project.gif)
+
+<a name="api-user"></a>
+## Create API user
+The next step is to create a new user account, which is restricted to the project.  
+This ensures that the API access is limited to that project.
+
+![Create API user](doc/otc_2_user.gif)
+
+<a name="key-pair"></a>
+## Import Key Pair
+:warning: Now log in with the newly created API user account and select your project.
+
+![Login as API user](doc/otc_3_login.gif)
+
+
+Import your SSH public key.
+
+![Import SSH Public Key](doc/otc_4_import_key.gif)
+
+<a name="vpc-subnet-securitygroup"></a>
+## Create VPC, Subnet and Security Group
+- VPC (Virtual Private Cloud) and Subnet:
+
+![Create VPC and Subnet](doc/otc_5_vpc_subnet.gif)
+
+- Security Group:  
+The configured Security Group should allow all incoming TCP / UDP traffic.  
+If you want to secure the management interfaces, you can limit the incoming "allow all" traffic to the port range of 1-64000 and allow access to ports > 64000 only from your trusted IPs.
+
+![Create Security Group](doc/otc_6_sec_group.gif)
+
+<a name="clone-git"></a>
+# Clone Git Repository
+Clone the `tpotce` repository to your Ansible Master:  
+`git clone https://github.com/dtag-dev-sec/tpotce.git`  
+All Ansible related files are located in the [`cloud/ansible/openstack`](../../cloud/ansible/openstack) folder.
+
+<a name="settings"></a>
+# Settings and recommended values
+You can configure all aspects of your Elastic Cloud Server and T-Pot before using the Playbook.  
+The settings are located in the following Ansible vars files:
+
+<a name="os-auth"></a>
+## OpenStack authentication variables
+Located at [`openstack/roles/deploy/vars/os_auth.yaml`](openstack/roles/deploy/vars/os_auth.yaml).  
+Enter your Open Telekom Cloud API user credentials here (username, password, project name, user domain name):  
+```
+auth_url: https://iam.eu-de.otc.t-systems.com/v3
+username: your_api_user
+password: your_password
+project_name: eu-de_your_project
+os_user_domain_name: OTC-EU-DE-000000000010000XXXXX
+```
+You can also perform different authentication methods like sourcing your `.ostackrc` file or using the OpenStack `clouds.yaml` file.  
+For more information have a look in the [os_server](https://docs.ansible.com/ansible/latest/modules/os_server_module.html) Ansible module documentation.
+
+<a name="remote-user"></a>
+## Ansible remote user
+You may have to adjust the `remote_user` in the Ansible Playbook under [`openstack/deploy_tpot.yaml`](openstack/deploy_tpot.yaml) depending on your Debian base image (e.g. on Open Telekom Cloud the default Debian user is `linux`).
+
+<a name="instance-settings"></a>
+## Instance settings
+Located at [`openstack/roles/deploy/vars/main.yaml`](openstack/roles/deploy/vars/main.yaml).  
+Here you can customize your virtual machine specifications:
+  - Specify the region name
+  - Choose an availability zone. For Open Telekom Cloud reference see [here](https://docs.otc.t-systems.com/en-us/endpoint/index.html).
+  - Change the OS image (For T-Pot we need Debian 9)
+  - (Optional) Change the volume size
+  - Specify your key pair
+  - (Optional) Change the instance type (flavor)  
+    `s2.medium.8` corresponds to 1 vCPU and 8GB of RAM and is the minimum required flavor.  
+    A full list of Open telekom Cloud flavors can be found [here](https://docs.otc.t-systems.com/en-us/usermanual/ecs/en-us_topic_0035470096.html).
+  - Specify the security group
+  - Specify the network ID (For Open Telekom Cloud you can find the ID in the Web Console under `Virtual Private Cloud --> your-vpc --> your-subnet --> Network ID`; In general for OpenStack clouds you can use the `python-openstackclient` to retrieve information about your resources)
+
+```
+region_name: eu-de
+availability_zone: eu-de-03
+image: Standard_Debian_9_latest
+volume_size: 128
+key_name: your-KeyPair
+flavor: s2.medium.8
+security_groups: your-sg
+network: your-network-id
+```
+
+<a name="user-password"></a>
+## User password
+Located at [`openstack/roles/install/vars/main.yaml`](openstack/roles/install/vars/main.yaml).  
+Here you can set the password for your Debian user (**you should definitely change that**).
+```
+user_password: LiNuXuSeRPaSs#
+```
+
+<a name="tpot-conf"></a>
+## Configure `tpot.conf.dist`
+The file is located in [`iso/installer/tpot.conf.dist`](../../iso/installer/tpot.conf.dist).  
+Here you can choose:
+  - between the various T-Pot editions
+  - a username for the web interface
+  - a password for the web interface (**you should definitely change that**)
+
+```
+# tpot configuration file
+# myCONF_TPOT_FLAVOR=[STANDARD, SENSOR, INDUSTRIAL, COLLECTOR, NEXTGEN]
+myCONF_TPOT_FLAVOR='STANDARD'
+myCONF_WEB_USER='webuser'
+myCONF_WEB_PW='w3b$ecret'
+```
+
+<a name="ews-cfg"></a>
+## Optional: Custom `ews.cfg`
+Enable this by uncommenting the role in the [deploy_tpot.yaml](openstack/deploy_tpot.yaml) playbook.
+```
+#    - custom_ews
+```
+
+You can use a custom config file for `ewsposter`.  
+e.g. when you have your own credentials for delivering data to our [Sicherheitstacho](https://sicherheitstacho.eu/start/main).  
+You can find the `ews.cfg` template file here: [`openstack/roles/custom_ews/templates/ews.cfg`](openstack/roles/custom_ews/templates/ews.cfg) and adapt it for your needs.
+
+For setting custom credentials, these settings would be relevant for you (the rest of the file can stay as is):  
+```
+[MAIN]
+...
+contact = your_email_address
+...
+
+[EWS]
+...
+username = your_username
+token = your_token
+...
+```
+
+<a name="hpfeeds"></a>
+## Optional: Custom HPFEEDS
+Enable this by uncommenting the role in the [deploy_tpot.yaml](openstack/deploy_tpot.yaml) playbook.
+```
+#    - custom_hpfeeds
+```
+
+You can specify custom HPFEEDS in [`openstack/roles/custom_hpfeeds/templates/hpfeeds.cfg`](openstack/roles/custom_hpfeeds/templates/hpfeeds.cfg).  
+That file contains the defaults (turned off) and you can adapt it for your needs, e.g. for SISSDEN:
+```
+myENABLE=true
+myHOST=hpfeeds.sissden.eu
+myPORT=10000
+myCHANNEL=t-pot.events
+myCERT=/opt/ewsposter/sissden.pem
+myIDENT=your_user
+mySECRET=your_secret
+myFORMAT=json
+```
+
+<a name="deploy"></a>
+# Deploying a T-Pot :honey_pot::honeybee:
+Now, after configuring everything, we can finally start deploying T-Pots!  
+Go to the [`openstack`](openstack) folder and run the Ansible Playbook with:  
+`ansible-playbook deploy_tpot.yaml`  
+(Yes, it is as easy as that :smile:)
+
+If you are running on a machine which asks for a sudo password, you can use:  
+`ansible-playbook --ask-become-pass deploy_tpot.yaml`
+
+The Playbook will first install required packages on the Ansible Master and then deploy a new server instance.  
+After that, T-Pot gets installed and configured on the newly created host, optionally custom configs are applied and finally it reboots.
+
+<a name="documentation"></a>
+# Further documentation
+- [Ansible Documentation](https://docs.ansible.com/ansible/latest/)
+- [Cloud modules — Ansible Documentation](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html)
+- [os_server – Create/Delete Compute Instances from OpenStack — Ansible Documentation](https://docs.ansible.com/ansible/latest/modules/os_server_module.html)
+- [Open Telekom Cloud Help Center](https://docs.otc.t-systems.com/)
+- [Open Telekom Cloud API Overview](https://docs.otc.t-systems.com/en-us/api/wp/en-us_topic_0052070394.html)

cloud/ansible/openstack/ansible.cfg

@@ -0,0 +1,5 @@
+[defaults]
+host_key_checking = false
+
+[ssh_connection]
+scp_if_ssh = true

cloud/ansible/openstack/deploy_tpot.yaml

@@ -0,0 +1,25 @@
+- name: Check host prerequisites
+  hosts: localhost
+  become: yes
+  become_user: root
+  become_method: sudo
+  roles:
+    - check
+
+- name: Deploy instance
+  hosts: localhost
+  roles:
+    - deploy
+
+- name: Install T-Pot on new instance
+  hosts: TPOT
+  remote_user: linux
+  become: yes
+  become_user: root
+  become_method: sudo
+  gather_facts: no
+  roles:
+    - install
+#    - custom_ews
+#    - custom_hpfeeds
+    - reboot

cloud/ansible/openstack/roles/check/tasks/main.yaml

@@ -0,0 +1,28 @@
+- name: Install pwgen
+  package:
+    name: pwgen
+    state: present
+
+- name: Install setuptools
+  package:
+    name: python-setuptools
+    state: present
+
+- name: Install pip
+  package:
+    name: python-pip
+    state: present
+
+- name: Install openstacksdk
+  pip:
+    name: openstacksdk
+
+- name: Set fact for agent forwarding
+  set_fact:
+    agent_forwarding: "{{ lookup('env','SSH_AUTH_SOCK') }}"
+
+- name: Check if agent forwarding is enabled
+  fail:
+    msg: Please enable agent forwarding to allow Ansible to connect to the remote host!
+  ignore_errors: yes
+  when: agent_forwarding == ""

cloud/ansible/openstack/roles/custom_ews/tasks/main.yaml

@@ -0,0 +1,13 @@
+- name: Copy ews configuration file
+  template:
+    src: ../templates/ews.cfg
+    dest: /data/ews/conf
+    owner: root
+    group: root
+    mode: 0644
+
+- name: Patching tpot.yml with custom ews configuration file
+  lineinfile:
+    path: /opt/tpot/etc/tpot.yml
+    insertafter: '/opt/ewsposter/ews.ip'
+    line: '     - /data/ews/conf/ews.cfg:/opt/ewsposter/ews.cfg'

cloud/ansible/openstack/roles/custom_ews/templates/ews.cfg

@@ -0,0 +1,137 @@
+[MAIN]
+homedir = /opt/ewsposter/
+spooldir = /opt/ewsposter/spool/
+logdir = /opt/ewsposter/log/
+del_malware_after_send = false
+send_malware = true
+sendlimit = 500
+contact = your_email_address
+proxy =
+ip =
+
+[EWS]
+ews = true
+username = your_username
+token = your_token
+rhost_first = https://community.sicherheitstacho.eu/ews-0.1/alert/postSimpleMessage
+rhost_second = https://community.sicherheitstacho.eu/ews-0.1/alert/postSimpleMessage
+ignorecert = false
+
+[HPFEED]
+hpfeed = %(EWS_HPFEEDS_ENABLE)s
+host = %(EWS_HPFEEDS_HOST)s
+port = %(EWS_HPFEEDS_PORT)s
+channels = %(EWS_HPFEEDS_CHANNELS)s
+ident = %(EWS_HPFEEDS_IDENT)s
+secret= %(EWS_HPFEEDS_SECRET)s
+# path/to/certificate for tls broker - or "false" for non-tls broker
+tlscert = %(EWS_HPFEEDS_TLSCERT)s
+# hpfeeds submission format: "ews" (xml) or "json"
+hpfformat = %(EWS_HPFEEDS_FORMAT)s
+
+[EWSJSON]
+json = false
+jsondir = /data/ews/json/
+
+[GLASTOPFV3]
+glastopfv3 = true
+nodeid = glastopfv3-{{ ansible_hostname }}
+sqlitedb = /data/glastopf/db/glastopf.db
+malwaredir = /data/glastopf/data/files/
+
+[GLASTOPFV2]
+glastopfv2 = false
+nodeid =
+mysqlhost =
+mysqldb =
+mysqluser =
+mysqlpw =
+malwaredir =
+
+[KIPPO]
+kippo = false
+nodeid =
+mysqlhost =
+mysqldb =
+mysqluser =
+mysqlpw =
+malwaredir =
+
+[COWRIE]
+cowrie = true
+nodeid = cowrie-{{ ansible_hostname }}
+logfile = /data/cowrie/log/cowrie.json
+
+[DIONAEA]
+dionaea = true
+nodeid = dionaea-{{ ansible_hostname }}
+malwaredir = /data/dionaea/binaries/
+sqlitedb = /data/dionaea/log/dionaea.sqlite
+
+[HONEYTRAP]
+honeytrap = true
+nodeid = honeytrap-{{ ansible_hostname }}
+newversion = true
+payloaddir = /data/honeytrap/attacks/
+attackerfile = /data/honeytrap/log/attacker.log
+
+[RDPDETECT]
+rdpdetect = false
+nodeid =
+iptableslog =
+targetip =
+
+[EMOBILITY]
+eMobility = false
+nodeid = emobility-{{ ansible_hostname }}
+logfile = /data/emobility/log/centralsystemEWS.log
+
+[CONPOT]
+conpot = true
+nodeid = conpot-{{ ansible_hostname }}
+logfile = /data/conpot/log/conpot*.json
+
+[ELASTICPOT]
+elasticpot = true
+nodeid = elasticpot-{{ ansible_hostname }}
+logfile = /data/elasticpot/log/elasticpot.log
+
+[SURICATA]
+suricata = true
+nodeid = suricata-{{ ansible_hostname }}
+logfile = /data/suricata/log/eve.json
+
+[MAILONEY]
+mailoney = true
+nodeid = mailoney-{{ ansible_hostname }}
+logfile = /data/mailoney/log/commands.log
+
+[RDPY]
+rdpy = true
+nodeid = rdpy-{{ ansible_hostname }}
+logfile = /data/rdpy/log/rdpy.log
+
+[VNCLOWPOT]
+vnclowpot = true
+nodeid = vnclowpot-{{ ansible_hostname }}
+logfile = /data/vnclowpot/log/vnclowpot.log
+
+[HERALDING]
+heralding = true
+nodeid = heralding-{{ ansible_hostname }}
+logfile = /data/heralding/log/auth.csv
+
+[CISCOASA]
+ciscoasa = true
+nodeid = ciscoasa-{{ ansible_hostname }}
+logfile = /data/ciscoasa/log/ciscoasa.log
+
+[TANNER]
+tanner = true
+nodeid = tanner-{{ ansible_hostname }}
+logfile = /data/tanner/log/tanner_report.json
+
+[GLUTTON]
+glutton = true
+nodeid = glutton-{{ ansible_hostname }}
+logfile = /data/glutton/log/glutton.log

cloud/ansible/openstack/roles/custom_hpfeeds/tasks/main.yaml

@@ -0,0 +1,10 @@
+- name: Copy hpfeeds configuration file
+  template:
+    src: ../templates/hpfeeds.cfg
+    dest: /data/ews/conf
+    owner: root
+    group: root
+    mode: 0644
+
+- name: Applying hpfeeds settings
+  command: /opt/tpot/bin/hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg

cloud/ansible/openstack/roles/custom_hpfeeds/templates/hpfeeds.cfg

@@ -0,0 +1,8 @@
+myENABLE=false
+myHOST=host
+myPORT=port
+myCHANNEL=channels
+myCERT=false
+myIDENT=user
+mySECRET=secret
+myFORMAT=json

cloud/ansible/openstack/roles/deploy/tasks/main.yaml

@@ -0,0 +1,34 @@
+- name: Create T-Pot name
+  shell: echo t-pot-ansible-$(pwgen -ns 6 -1)
+  register: tpot_name
+
+- name: Import OpenStack authentication variables
+  include_vars:
+    file: roles/deploy/vars/os_auth.yaml
+
+- name: Launch an instance
+  os_server:
+    auth:
+      auth_url: "{{ auth_url }}"
+      username: "{{ username }}"
+      password: "{{ password }}"
+      project_name: "{{ project_name }}"
+      os_user_domain_name: "{{ os_user_domain_name }}"
+    name: "{{ tpot_name.stdout }}"
+    region_name: "{{ region_name }}"
+    availability_zone: "{{ availability_zone }}"
+    image: "{{ image }}"
+    boot_from_volume: yes
+    volume_size: "{{ volume_size }}"
+    key_name: "{{ key_name }}"
+    timeout: 200
+    flavor: "{{ flavor }}"
+    security_groups: "{{ security_groups }}"
+    network: "{{ network }}"
+  register: tpot
+
+- name: Add instance to inventory
+  add_host:
+    hostname: "{{ tpot_name.stdout }}"
+    ansible_host: "{{ tpot.server.public_v4 }}"
+    groups: TPOT

cloud/ansible/openstack/roles/deploy/vars/main.yaml

@@ -0,0 +1,8 @@
+region_name: eu-de
+availability_zone: eu-de-03
+image: Standard_Debian_9_latest
+volume_size: 128
+key_name: your-KeyPair
+flavor: s2.medium.8
+security_groups: your-sg
+network: your-network-id

cloud/ansible/openstack/roles/deploy/vars/os_auth.yaml

@@ -0,0 +1,5 @@
+auth_url: https://iam.eu-de.otc.t-systems.com/v3
+username: your_api_user
+password: your_password
+project_name: eu-de_your_project
+os_user_domain_name: OTC-EU-DE-000000000010000XXXXX

cloud/ansible/openstack/roles/install/tasks/main.yaml

@@ -0,0 +1,51 @@
+- name: Waiting for SSH connection
+  wait_for_connection:
+    delay: 30
+    timeout: 300
+
+- name: Gathering facts
+  setup:
+
+- name: Cloning T-Pot install directory
+  git:
+    repo: "https://github.com/dtag-dev-sec/tpotce.git"
+    dest: /root/tpot
+
+- name: Prepare to set user password 
+  set_fact:
+    user_name: "{{ ansible_user }}"
+    user_password: "{{ user_password }}"
+    user_salt: "s0mew1ck3dTpoT"
+
+- name: Changing password for user {{ user_name }} to {{ user_password }}
+  user:
+   name: "{{ ansible_user }}"
+   password: "{{ user_password | password_hash('sha512', user_salt) }}"
+   state: present
+   shell: /bin/bash
+   update_password: always
+
+- name: Copy T-Pot configuration file
+  template:
+    src: ../../../../../../iso/installer/tpot.conf.dist
+    dest: /root/tpot.conf
+    owner: root
+    group: root
+    mode: 0644
+
+- name: Install T-Pot on instance -  be patient, this might take 15 to 30 minutes depending on the connection speed. No further output is given.
+  command: /root/tpot/iso/installer/install.sh --type=auto --conf=/root/tpot.conf
+
+- name: Delete T-Pot configuration file
+  file:
+    path: /root/tpot.conf
+    state: absent
+
+- name: Change unattended-upgrades to take default action
+  blockinfile:
+    dest: /etc/apt/apt.conf.d/50unattended-upgrades
+    block: |
+      Dpkg::Options {
+        "--force-confdef";
+        "--force-confold";
+      }

cloud/ansible/openstack/roles/install/vars/main.yaml

@@ -0,0 +1 @@
+user_password: LiNuXuSeRPaSs#

cloud/ansible/openstack/roles/reboot/tasks/main.yaml

@@ -0,0 +1,15 @@
+- name: Finally rebooting T-Pot in one minute
+  shell: /sbin/shutdown -r -t 1
+  become: true
+
+- name: Next login options
+  debug:
+    msg:
+    - "*****    SSH Access:"
+    - "*****    ssh {{ ansible_user }}@{{ ansible_host }} -p 64295"
+    - ""
+    - "*****    Web UI:"
+    - "*****    https://{{ ansible_host }}:64297"
+    - ""
+    - "*****    Admin UI:"
+    - "*****    https://{{ ansible_host }}:64294"

cloud/terraform/.gitignore

@@ -0,0 +1,2 @@
+**/.terraform
+**/terraform.*

cloud/terraform/README.md

@@ -0,0 +1,131 @@
+# T-Pot Terraform
+
+This [Terraform](https://www.terraform.io/) configuration can be used to provision a T-Pot instance in AWS in addition to all of the necessary pre-requisites. Specifically, the following resources will be created:
+
+* EC2 instance:
+  * t3.large (2 vCPU, 8 GiB RAM)
+  * 128GB disk
+  * [Debian Stretch](https://wiki.debian.org/Cloud/AmazonEC2Image/Stretch) (The T-Pot installation script will then upgrade this to Debian Sid)
+* AWS Security Group:
+  * TCP/UDP ports <= 64000 open to the Internet
+  * TCP ports 64294, 64295 and 64297 open to a chosen administrative IP
+
+[Cloud-init](https://cloudinit.readthedocs.io/en/latest/) is used to bootstrap the instance and install T-Pot on startup. Additional provisioning using Ansible etc. is not required.
+
+The following resources are NOT automatically created and need to be specified in the configuration below:
+
+* VPC
+* Subnet
+
+## Pre-Requisites
+
+* [Terraform](https://www.terraform.io/) 0.12
+* AWS Account
+  * Existing VPC. VPC ID should be specified in configuration below
+  * Existing subnet. Subnet ID should be specified in configuration below
+* AWS Authentication credentials should be [set using environment variables](https://www.terraform.io/docs/providers/aws/index.html#environment-variables)
+
+## Required Configuration Changes
+
+### Terraform Variables
+
+In `aws/variables.tf`, change the following variables to correspond to your existing EC2 infrastructure:
+
+* `admin_ip` - source IP address(es) that you will use to administer the system. Connections to TCP ports 64294, 64295 and 64297 will be allowed from this IP only. Multiple IPs or CIDR blocks can be specified in the format: `["127.0.0.1/32", "192.168.0.0/24"]`
+* `ec2_vpc_id`
+* `ec2_subnet_id`
+* `ec2_region`
+
+### Admin Credentials
+
+In `tpot.conf`, change the following variables:
+
+```
+myCONF_WEB_USER='webuser'
+myCONF_WEB_PW='w3b$ecret'
+```
+
+This will be used to configure credentials for the T-Pot Kibana interface. Refer to [Options](https://github.com/dtag-dev-sec/tpotce#options) for more information.
+
+## Initialising
+
+The [`terraform init`](https://www.terraform.io/docs/commands/init.html) command is used to initialize a working directory containing Terraform configuration files.
+
+```
+$ cd aws
+$ terraform init
+
+Initializing the backend...
+
+Initializing provider plugins...
+- Checking for available provider plugins...
+- Downloading plugin for provider "aws" (terraform-providers/aws) 2.16.0...
+
+The following providers do not have any version constraints in configuration,
+so the latest version was installed.
+
+To prevent automatic upgrades to new major versions that may contain breaking
+changes, it is recommended to add version = "..." constraints to the
+corresponding provider blocks in configuration, with the constraint strings
+suggested below.
+
+* provider.aws: version = "~> 2.16"
+
+Terraform has been successfully initialized!
+
+You may now begin working with Terraform. Try running "terraform plan" to see
+any changes that are required for your infrastructure. All Terraform commands
+should now work.
+
+If you ever set or change modules or backend configuration for Terraform,
+rerun this command to reinitialize your working directory. If you forget, other
+commands will detect it and remind you to do so if necessary.
+```
+
+## Applying the Configuration
+
+The [`terraform apply`](https://www.terraform.io/docs/commands/apply.html) command is used to apply the changes required to reach the desired state of the configuration, or the pre-determined set of actions generated by a [`terraform plan`](https://www.terraform.io/docs/commands/plan.html) execution plan.
+
+```
+$ terraform apply
+
+An execution plan has been generated and is shown below.
+Resource actions are indicated with the following symbols:
+  + create
+
+Terraform will perform the following actions:
+
+  # aws_instance.tpot will be created
+  ...
+
+  # aws_security_group.tpot will be created
+  ...
+
+Plan: 2 to add, 0 to change, 0 to destroy.
+
+Do you want to perform these actions?
+  Terraform will perform the actions described above.
+  Only 'yes' will be accepted to approve.
+
+  Enter a value:
+```
+
+This will perform the following actions:
+
+1. Create EC2 security group
+2. Start a Debian EC2 instance
+3. Update all packages and reboot if necessary
+4. Install T-Pot and required dependencies
+5. Reboot
+
+## Connecting to the Instance
+
+### SSH
+
+Prior to the final reboot, you will temporarily be able to SSH to port 22 as per standard. Following the reboot, port 22 is used for the honeypot. The *real* SSH server is listening on port **64295**
+
+### Browser
+
+https://www.example.com:64297/
+
+Replace with the FQDN of your EC2 instance. Refer to the [T-POT documentation](https://github.com/dtag-dev-sec/tpotce#ssh-and-web-access) for further details.

cloud/terraform/aws/main.tf

@@ -0,0 +1,65 @@
+provider "aws" {
+  region = var.ec2_region
+}
+
+resource "aws_security_group" "tpot" {
+  name        = "T-Pot"
+  description = "T-Pot Honeypot"
+  vpc_id      = var.ec2_vpc_id
+  ingress {
+    from_port   = 0
+    to_port     = 64000
+    protocol    = "tcp"
+    cidr_blocks = ["0.0.0.0/0"]
+  }
+  ingress {
+    from_port   = 0
+    to_port     = 64000
+    protocol    = "udp"
+    cidr_blocks = ["0.0.0.0/0"]
+  }
+  ingress {
+    from_port   = 64294
+    to_port     = 64294
+    protocol    = "tcp"
+    cidr_blocks = var.admin_ip
+  }
+  ingress {
+    from_port   = 64295
+    to_port     = 64295
+    protocol    = "tcp"
+    cidr_blocks = var.admin_ip
+  }
+  ingress {
+    from_port   = 64297
+    to_port     = 64297
+    protocol    = "tcp"
+    cidr_blocks = var.admin_ip
+  }
+  egress {
+    from_port   = 0
+    to_port     = 0
+    protocol    = "-1"
+    cidr_blocks = ["0.0.0.0/0"]
+  }
+  tags = {
+    Name = "T-Pot"
+  }
+}
+
+resource "aws_instance" "tpot" {
+  ami           = var.ec2_ami[var.ec2_region]
+  instance_type = var.ec2_instance_type
+  key_name      = var.ec2_ssh_key_name
+  subnet_id     = var.ec2_subnet_id
+  tags = {
+    Name = "T-Pot Honeypot"
+  }
+  root_block_device {
+    volume_type           = "gp2"
+    volume_size           = 128
+    delete_on_termination = true
+  }
+  user_data              = "${file("../cloud-init.yaml")}    content: ${base64encode(file("../tpot.conf"))}"
+  vpc_security_group_ids = [aws_security_group.tpot.id]
+}

cloud/terraform/aws/outputs.tf

@@ -0,0 +1,12 @@
+output "Admin_UI" {
+  value = "https://${aws_instance.tpot.public_dns}:64294/"
+}
+
+output "SSH_Access" {
+  value = "ssh -i {private_key_file} -p 64295 admin@${aws_instance.tpot.public_dns}"
+}
+
+output "Web_UI" {
+  value = "https://${aws_instance.tpot.public_dns}:64297/"
+}
+

cloud/terraform/aws/variables.tf

@@ -0,0 +1,53 @@
+variable "admin_ip" {
+  default     = ["127.0.0.1/32"]
+  description = "admin IP addresses in CIDR format"
+}
+
+variable "ec2_vpc_id" {
+  description = "ID of AWS VPC"
+  default     = "vpc-XXX"
+}
+
+variable "ec2_subnet_id" {
+  description = "ID of AWS VPC subnet"
+  default     = "subnet-YYY"
+}
+
+variable "ec2_region" {
+  description = "AWS region to launch servers"
+  default     = "eu-west-1"
+}
+
+variable "ec2_ssh_key_name" {
+  default = "default"
+}
+
+# https://aws.amazon.com/ec2/instance-types/
+# t3.large = 2 vCPU, 8 GiB RAM
+variable "ec2_instance_type" {
+  default = "t3.large"
+}
+
+# Refer to https://wiki.debian.org/Cloud/AmazonEC2Image/Stretch
+variable "ec2_ami" {
+  type = map(string)
+  default = {
+    "ap-northeast-1" = "ami-09fbcd30452841cb9"
+    "ap-northeast-2" = "ami-08363ccce96df1fff"
+    "ap-south-1"     = "ami-0dc98cbb0d0e49162"
+    "ap-southeast-1" = "ami-0555b1a5444087dd4"
+    "ap-southeast-2" = "ami-029c54f988446691a"
+    "ca-central-1"   = "ami-04413a263a7d94982"
+    "eu-central-1"   = "ami-01fb3b7bab31acac5"
+    "eu-north-1"     = "ami-050f04ca573daa1fb"
+    "eu-west-1"      = "ami-0968f6a31fc6cffc0"
+    "eu-west-2"      = "ami-0faa9c9b5399088fd"
+    "eu-west-3"      = "ami-0cd23820af84edc85"
+    "sa-east-1"      = "ami-030580e61468e54bd"
+    "us-east-1"      = "ami-0357081a1383dc76b"
+    "us-east-2"      = "ami-09c10a66337c79669"
+    "us-west-1"      = "ami-0adbaf2e0ce044437"
+    "us-west-2"      = "ami-05a3ef6744aa96514"
+  }
+}
+

cloud/terraform/aws/versions.tf

@@ -0,0 +1,3 @@
+terraform {
+  required_version = ">= 0.12"
+}

cloud/terraform/cloud-init.yaml

@@ -0,0 +1,25 @@
+#cloud-config
+timezone: UTC
+
+package_update: true
+package_upgrade: true
+package_reboot_if_required: true
+
+packages:
+  - git
+
+runcmd:
+  - git clone https://github.com/dtag-dev-sec/tpotce /root/tpot
+  - /root/tpot/iso/installer/install.sh --type=auto --conf=/root/tpot.conf
+  - rm /root/tpot.conf
+  - /sbin/shutdown -r +5
+
+# The contents of tpot.conf will be base64 encoded and appended to this file
+# via the terraform configuration in main.tf
+#
+# Make sure there are no trailing new lines after "permissions" below
+write_files:
+  - encoding: b64
+    owner: root:root
+    path: /root/tpot.conf
+    permissions: '0600'

cloud/terraform/tpot.conf

@@ -0,0 +1,5 @@
+# tpot configuration file
+# myCONF_TPOT_FLAVOR=[STANDARD, SENSOR, INDUSTRIAL, COLLECTOR, NEXTGEN]
+myCONF_TPOT_FLAVOR='STANDARD'
+myCONF_WEB_USER='webuser'
+myCONF_WEB_PW='w3b$ecret'

docker/adbhoney/Dockerfile

@@ -0,0 +1,31 @@
+FROM alpine 
+
+# Install packages
+RUN apk -U --no-cache add \
+            git \
+            libcap \
+            python \
+            python-dev && \
+
+# Install adbhoney from git
+    git clone --depth=1 https://github.com/huuck/ADBHoney /opt/adbhoney && \
+    sed -i 's/dst_ip/dest_ip/' /opt/adbhoney/main.py && \
+    sed -i 's/dst_port/dest_port/' /opt/adbhoney/main.py && \
+
+# Setup user, groups and configs
+    addgroup -g 2000 adbhoney && \
+    adduser -S -H -s /bin/ash -u 2000 -D -g 2000 adbhoney && \
+    chown -R adbhoney:adbhoney /opt/adbhoney && \
+    setcap cap_net_bind_service=+ep /usr/bin/python2.7 && \
+
+# Clean up
+    apk del --purge git \
+                    python-dev && \
+    rm -rf /root/* && \
+    rm -rf /var/cache/apk/*
+
+# Set workdir and start adbhoney
+STOPSIGNAL SIGINT
+USER adbhoney:adbhoney
+WORKDIR /opt/adbhoney/
+CMD nohup /usr/bin/python main.py -l log/adbhoney.log -j log/adbhoney.json -d dl/

docker/adbhoney/docker-compose.yml

@@ -0,0 +1,21 @@
+version: '2.3'
+
+networks:
+  adbhoney_local:
+
+services:
+
+# Adbhoney service
+  adbhoney:
+    build: .
+    container_name: adbhoney
+    restart: always
+    networks:
+     - adbhoney_local
+    ports:
+     - "5555:5555"
+    image: "dtagdevsec/adbhoney:1903"
+    read_only: true
+    volumes:
+     - /data/adbhoney/log:/opt/adbhoney/log
+     - /data/adbhoney/downloads:/opt/adbhoney/dl

docker/ciscoasa/Dockerfile

@@ -0,0 +1,44 @@
+FROM alpine
+
+# Include dist
+ADD dist/ /root/dist/
+
+# Setup env and apt
+RUN apk -U upgrade && \
+    apk add build-base \
+            git \
+            libffi \
+            libffi-dev \
+            openssl \
+            openssl-dev \
+            python3 \
+            python3-dev && \
+
+# Setup user
+    addgroup -g 2000 ciscoasa && \
+    adduser -S -s /bin/bash -u 2000 -D -g 2000 ciscoasa && \
+
+# Get and install packages
+    mkdir -p /opt/ && \
+    cd /opt/ && \
+    git clone --depth=1 https://github.com/cymmetria/ciscoasa_honeypot && \
+    cd ciscoasa_honeypot && \
+    pip3 install --no-cache-dir --upgrade pip && \
+    pip3 install --no-cache-dir -r requirements.txt && \
+    cp /root/dist/asa_server.py /opt/ciscoasa_honeypot && \
+    chown -R ciscoasa:ciscoasa /opt/ciscoasa_honeypot && \
+
+# Clean up
+    apk del --purge build-base \
+                    git \
+                    libffi-dev \
+                    openssl-dev \
+                    python3-dev && \
+    rm -rf /root/* && \
+    rm -rf /var/cache/apk/*
+
+# Start ciscoasa
+STOPSIGNAL SIGINT
+WORKDIR /tmp/ciscoasa/
+USER ciscoasa:ciscoasa
+CMD cp -R /opt/ciscoasa_honeypot/* /tmp/ciscoasa && exec python3 asa_server.py --ike-port 5000 --enable_ssl --port 8443 --verbose >> /var/log/ciscoasa/ciscoasa.log 2>&1

docker/ciscoasa/README.md

@@ -0,0 +1,15 @@
+[![](https://images.microbadger.com/badges/version/dtagdevsec/ciscoasa:1903.svg)](https://microbadger.com/images/dtagdevsec/ciscoasa:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/ciscoasa:1903.svg)](https://microbadger.com/images/dtagdevsec/ciscoasa:1903 "Get your own image badge on microbadger.com")
+
+# ciscoasa
+
+[Ciscoasa](https://github.com/cymmetria/ciscoasa_honeypot) is a low interaction honeypot for the Cisco ASA component capable of detecting CVE-2018-0101, a DoS and remote code execution vulnerability
+
+This dockerized version is part of the **[T-Pot community honeypot](http://dtag-dev-sec.github.io/)** of Deutsche Telekom AG.
+
+The `Dockerfile` contains the blueprint for the dockerized ciscoasa and will be used to setup the docker image.
+
+The `docker-compose.yml` contains the necessary settings to test conpot using `docker-compose`. This will ensure to start the docker container with the appropriate permissions and port mappings.
+
+# Ciscoasa Dashboard
+
+![Ciscoasa Dashboard](doc/dashboard.png)

docker/ciscoasa/dist/asa_server.py

@@ -0,0 +1,307 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+import os
+import time
+import socket
+import logging
+logging.basicConfig(format='%(message)s')
+import threading
+from io import BytesIO
+from xml.etree import ElementTree
+from http.server import HTTPServer
+from socketserver import ThreadingMixIn
+from http.server import SimpleHTTPRequestHandler
+import ike_server
+import datetime
+
+
+class NonBlockingHTTPServer(ThreadingMixIn, HTTPServer):
+    pass
+
+class hpflogger:
+    def __init__(self, hpfserver, hpfport, hpfident, hpfsecret, hpfchannel, serverid, verbose):
+        self.hpfserver=hpfserver
+        self.hpfport=hpfport
+        self.hpfident=hpfident
+        self.hpfsecret=hpfsecret
+        self.hpfchannel=hpfchannel
+        self.serverid=serverid
+        self.hpc=None
+        self.verbose=verbose
+        if (self.hpfserver and self.hpfport and self.hpfident and self.hpfport and self.hpfchannel and self.serverid):
+            import hpfeeds
+            try:
+                self.hpc = hpfeeds.new(self.hpfserver, self.hpfport, self.hpfident, self.hpfsecret)
+                logger.debug("Logging to hpfeeds using server: {0}, channel {1}.".format(self.hpfserver, self.hpfchannel))
+            except (hpfeeds.FeedException, socket.error, hpfeeds.Disconnect):
+                logger.critical("hpfeeds connection not successful")
+
+    def log(self, level, message):
+        if self.hpc:
+            if level in ['debug', 'info'] and not self.verbose:
+                return
+            self.hpc.publish(self.hpfchannel, "["+self.serverid+"] ["+level+"] ["+datetime.datetime.now().isoformat() +"] "  + str(message))
+
+
+def header_split(h):
+    return [list(map(str.strip, l.split(': ', 1))) for l in h.strip().splitlines()]
+
+
+class WebLogicHandler(SimpleHTTPRequestHandler):
+    logger = None
+    hpfl = None
+
+    protocol_version = "HTTP/1.1"
+
+    EXPLOIT_STRING = b"host-scan-reply"
+    RESPONSE = b"""<?xml version="1.0" encoding="UTF-8"?>
+<config-auth client="vpn" type="complete">
+<version who="sg">9.0(1)</version>
+<error id="98" param1="" param2="">VPN Server could not parse request.</error>
+</config-auth>"""
+
+    basepath = os.path.dirname(os.path.abspath(__file__))
+
+    alert_function = None
+
+    def setup(self):
+        SimpleHTTPRequestHandler.setup(self)
+        self.request.settimeout(3)
+
+    def send_header(self, keyword, value):
+        if keyword.lower() == 'server':
+            return
+        SimpleHTTPRequestHandler.send_header(self, keyword, value)
+
+    def send_head(self):
+        # send_head will return a file object that do_HEAD/GET will use
+        # do_GET/HEAD are already implemented by SimpleHTTPRequestHandler
+        filename = os.path.basename(self.path.rstrip('/').split('?', 1)[0])
+
+        if self.path == '/':
+            self.send_response(200)
+            for k, v in header_split("""
+                Content-Type: text/html
+                Cache-Control: no-cache
+                Pragma: no-cache
+                Set-Cookie: tg=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+                Set-Cookie: webvpn=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+                Set-Cookie: webvpnc=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+                Set-Cookie: webvpn_portal=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+                Set-Cookie: webvpnSharePoint=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+                Set-Cookie: webvpnlogin=1; path=/; secure
+                Set-Cookie: sdesktop=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+            """):
+                self.send_header(k, v)
+            self.end_headers()
+            return BytesIO(b'<html><script>document.location.replace("/+CSCOE+/logon.html")</script></html>\n')
+        elif filename == 'asa':  # don't allow dir listing
+            return self.send_file('wrong_url.html', 403)
+        else:
+            return self.send_file(filename)
+
+    def redirect(self, loc):
+        self.send_response(302)
+        for k, v in header_split("""
+            Content-Type: text/html
+            Content-Length: 0
+            Cache-Control: no-cache
+            Pragma: no-cache
+            Location: %s
+            Set-Cookie: tg=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+        """ % (loc,)):
+            self.send_header(k, v)
+        self.end_headers()
+
+    def do_GET(self):
+        if self.path == '/+CSCOE+/logon.html':
+            self.redirect('/+CSCOE+/logon.html?fcadbadd=1')
+            return
+        elif self.path.startswith('/+CSCOE+/logon.html?') and 'reason=1' in self.path:
+            self.wfile.write(self.send_file('logon_failure').getvalue())
+            return
+        SimpleHTTPRequestHandler.do_GET(self)
+
+    def do_POST(self):
+        data_len = int(self.headers.get('Content-length', 0))
+        data = self.rfile.read(data_len) if data_len else b''
+        body = self.RESPONSE
+        if self.EXPLOIT_STRING in data:
+            xml = ElementTree.fromstring(data)
+            payloads = []
+            for x in xml.iter('host-scan-reply'):
+                payloads.append(x.text)
+
+            self.alert_function(self.client_address[0], self.client_address[1], payloads)
+
+        elif self.path == '/':
+            self.redirect('/+webvpn+/index.html')
+            return
+        elif self.path == '/+CSCOE+/logon.html':
+            self.redirect('/+CSCOE+/logon.html?fcadbadd=1')
+            return
+        elif self.path.split('?', 1)[0] == '/+webvpn+/index.html':
+            with open(os.path.join(self.basepath, 'asa', "logon_redir.html"), 'rb') as fh:
+                body = fh.read()
+
+        self.send_response(200)
+        self.send_header('Content-Length', int(len(body)))
+        self.send_header('Content-Type', 'text/html; charset=UTF-8')
+        self.end_headers()
+        self.wfile.write(body)
+        return
+
+    def send_file(self, filename, status_code=200, headers=[]):
+        try:
+            with open(os.path.join(self.basepath, 'asa', filename), 'rb') as fh:
+                body = fh.read()
+                self.send_response(status_code)
+                for k, v in headers:
+                    self.send_header(k, v)
+                if status_code == 200:
+                    for k, v in header_split("""
+                        Cache-Control: max-age=0
+                        Set-Cookie: webvpn=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+                        Set-Cookie: webvpnc=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+                        Set-Cookie: webvpnlogin=1; secure
+                        X-Transcend-Version: 1
+                    """):
+                        self.send_header(k, v)
+                self.send_header('Content-Length', int(len(body)))
+                self.send_header('Content-Type', 'text/html')
+                self.end_headers()
+                return BytesIO(body)
+        except IOError:
+            return self.send_file('wrong_url.html', 404)
+
+    def log_message(self, format, *args):
+        self.logger.debug("{'timestamp': '%s', 'src_ip': '%s', 'payload_printable': '%s'}" %
+                          (datetime.datetime.now().isoformat(),
+                           self.client_address[0],
+                           format % args))
+        self.hpfl.log('debug', "%s - - [%s] %s" %
+                          (self.client_address[0],
+                           self.log_date_time_string(),
+                           format % args))
+
+    def handle_one_request(self):
+        """Handle a single HTTP request.
+        Overriden to not send 501 errors
+        """
+        self.close_connection = True
+        try:
+            self.raw_requestline = self.rfile.readline(65537)
+            if len(self.raw_requestline) > 65536:
+                self.requestline = ''
+                self.request_version = ''
+                self.command = ''
+                self.close_connection = 1
+                return
+            if not self.raw_requestline:
+                self.close_connection = 1
+                return
+            if not self.parse_request():
+                # An error code has been sent, just exit
+                return
+            mname = 'do_' + self.command
+            if not hasattr(self, mname):
+                self.log_request()
+                self.close_connection = True
+                return
+            method = getattr(self, mname)
+            method()
+            self.wfile.flush()  # actually send the response if not already done.
+        except socket.timeout as e:
+            # a read or a write timed out.  Discard this connection
+            self.log_error("Request timed out: %r", e)
+            self.close_connection = 1
+            return
+
+
+if __name__ == '__main__':
+    import click
+
+    logging.basicConfig(level=logging.INFO)
+    logger = logging.getLogger()
+    logger.info('info')
+
+    @click.command()
+    @click.option('-h', '--host', default='0.0.0.0', help='Host to listen')
+    @click.option('-p', '--port', default=8443, help='Port to listen', type=click.INT)
+    @click.option('-i', '--ike-port', default=5000, help='Port to listen for IKE', type=click.INT)
+    @click.option('-s', '--enable_ssl', default=False, help='Enable SSL', is_flag=True)
+    @click.option('-c', '--cert', default=None, help='Certificate File Path (will generate self signed '
+                                                     'cert if not supplied)')
+    @click.option('-v', '--verbose', default=False, help='Verbose logging', is_flag=True)
+
+    # hpfeeds options
+    @click.option('--hpfserver', default=os.environ.get('HPFEEDS_SERVER'), help='HPFeeds Server')
+    @click.option('--hpfport', default=os.environ.get('HPFEEDS_PORT'), help='HPFeeds Port', type=click.INT)
+    @click.option('--hpfident', default=os.environ.get('HPFEEDS_IDENT'), help='HPFeeds Ident')
+    @click.option('--hpfsecret', default=os.environ.get('HPFEEDS_SECRET'), help='HPFeeds Secret')
+    @click.option('--hpfchannel', default=os.environ.get('HPFEEDS_CHANNEL'), help='HPFeeds Channel')
+    @click.option('--serverid', default=os.environ.get('SERVERID'), help='Verbose logging')
+
+
+    def start(host, port, ike_port, enable_ssl, cert, verbose, hpfserver, hpfport, hpfident, hpfsecret, hpfchannel, serverid):
+        """
+           A low interaction honeypot for the Cisco ASA component capable of detecting CVE-2018-0101,
+           a DoS and remote code execution vulnerability
+        """
+
+        hpfl=hpflogger(hpfserver, hpfport, hpfident, hpfsecret, hpfchannel, serverid, verbose)
+
+        def alert(cls, host, port, payloads):
+            logger.critical({
+                 'timestamp': datetime.datetime.utcnow().isoformat(),
+                 'src_ip': host,
+                 'src_port': port,
+                 'payload_printable': payloads,
+            })
+            #log to hpfeeds
+            hpfl.log("critical", {
+                 'src': host,
+                 'spt': port,
+                 'data': payloads,
+             })
+
+        if verbose:
+            logger.setLevel(logging.DEBUG)
+
+        requestHandler = WebLogicHandler
+        requestHandler.alert_function = alert
+        requestHandler.logger = logger
+        requestHandler.hpfl = hpfl
+
+        def log_date_time_string():
+            """Return the current time formatted for logging."""
+            now = datetime.datetime.now().isoformat()
+            return now
+
+        def ike():
+            ike_server.start(host, ike_port, alert, logger, hpfl)
+        t = threading.Thread(target=ike)
+        t.daemon = True
+        t.start()
+
+        httpd = HTTPServer((host, port), requestHandler)
+        if enable_ssl:
+            import ssl
+            if not cert:
+                import gencert
+                cert = gencert.gencert()
+            httpd.socket = ssl.wrap_socket(httpd.socket, certfile=cert, server_side=True)
+
+        logger.info('Starting server on port {:d}/tcp, use <Ctrl-C> to stop'.format(port))
+        hpfl.log('info', 'Starting server on port {:d}/tcp, use <Ctrl-C> to stop'.format(port))
+
+        try:
+            httpd.serve_forever()
+        except KeyboardInterrupt:
+            pass
+        logger.info('Stopping server.')
+        hpfl.log('info', 'Stopping server.')
+
+        httpd.server_close()
+
+    start()

docker/ciscoasa/docker-compose.yml

@@ -0,0 +1,19 @@
+version: '2.3'
+
+services:
+
+# Ciscoasa service
+  ciscoasa:
+    build: .
+    container_name: ciscoasa
+    restart: always
+    tmpfs:
+     - /tmp/ciscoasa:uid=2000,gid=2000
+    network_mode: "host"
+    ports:
+     - "5000:5000/udp"
+     - "8443:8443"
+    image: "dtagdevsec/ciscoasa:1903"
+    read_only: true
+    volumes:
+     - /data/ciscoasa/log:/var/log/ciscoasa

docker/conpot/Dockerfile

@@ -1,43 +1,60 @@
 FROM alpine
-MAINTAINER MO
 
 # Include dist
 ADD dist/ /root/dist/
 
 # Setup apt
-RUN apk -U add bash \
+RUN apk -U add \
              build-base \
              file \
              git \
              libev \
              libtool \
+             libcap \
              libxslt \
              libxslt-dev \
              mariadb-dev \
-               mariadb-client-libs \
              pkgconfig \
-               python \
+             python3 \
-               python-dev \
+             python3-dev \
-               py-cffi && \
+             py-cffi \
+             py-cryptography \
+             tcpdump \
+             wget && \
 
 # Setup ConPot
-    git clone https://github.com/mushorg/conpot /opt/conpot/ && \
+    git clone --depth=1 https://github.com/mushorg/conpot /opt/conpot && \
     cd /opt/conpot/ && \
-    git checkout d97a68a054e4fe42ff90293188a5702ce8ab09a3 && \
+    # Patch to accept ENV for MIB path
-    cp /root/dist/requirements.txt /opt/conpot/ && \
+    sed -i "s/tmp_mib_dir = tempfile.mkdtemp()/tmp_mib_dir = tempfile.mkdtemp(dir=os.environ['CONPOT_TMP'])/" /opt/conpot/conpot/protocols/snmp/snmp_server.py && \
-    python setup.py install && \
+    # Change template default ports if <1024
+    sed -i 's/port="2121"/port="21"/' /opt/conpot/conpot/templates/default/ftp/ftp.xml && \ 
+    sed -i 's/port="8800"/port="80"/' /opt/conpot/conpot/templates/default/http/http.xml && \ 
+    sed -i 's/port="6230"/port="623"/' /opt/conpot/conpot/templates/default/ipmi/ipmi.xml && \ 
+    sed -i 's/port="5020"/port="502"/' /opt/conpot/conpot/templates/default/modbus/modbus.xml && \ 
+    sed -i 's/port="10201"/port="102"/' /opt/conpot/conpot/templates/default/s7comm/s7comm.xml && \ 
+    sed -i 's/port="16100"/port="161"/' /opt/conpot/conpot/templates/default/snmp/snmp.xml && \ 
+    sed -i 's/port="6969"/port="69"/' /opt/conpot/conpot/templates/default/tftp/tftp.xml && \ 
+    sed -i 's/port="16100"/port="161"/' /opt/conpot/conpot/templates/IEC104/snmp/snmp.xml && \ 
+    sed -i 's/port="6230"/port="623"/' /opt/conpot/conpot/templates/ipmi/ipmi/ipmi.xml && \ 
+    pip3 install --no-cache-dir -U pip setuptools && \
+    pip3 install --no-cache-dir . && \
     cd / && \
     rm -rf /opt/conpot /tmp/* /var/tmp/* && \
+    setcap cap_net_bind_service=+ep /usr/bin/python3.6 && \
     
-# Setup user, groups and configs
+# Get wireshark manuf db for scapy, setup configs, user, groups
+    mkdir -p /etc/conpot /var/log/conpot /usr/share/wireshark && \
+    wget https://github.com/wireshark/wireshark/raw/master/manuf -o /usr/share/wireshark/manuf && \
+    cp /root/dist/conpot.cfg /etc/conpot/conpot.cfg && \
+    cp -R /root/dist/templates /usr/lib/python3.6/site-packages/conpot/ && \
     addgroup -g 2000 conpot && \
-    adduser -S -s /bin/bash -u 2000 -D -g 2000 conpot && \
+    adduser -S -s /bin/ash -u 2000 -D -g 2000 conpot && \
-    mkdir -p /etc/conpot /var/log/conpot && \
-    mv /root/dist/conpot.cfg /etc/conpot/conpot.cfg && \
-    mv /root/dist/kamstrup_382/template.xml /usr/lib/python2.7/site-packages/Conpot-0.5.1-py2.7.egg/conpot/templates/kamstrup_382/ && \
 
 # Clean up
-    apk del build-base \
+    apk del --purge \
+            build-base \
+            cython-dev \
             file \
             git \
             libev \
@@ -45,10 +62,14 @@ RUN apk -U add bash \
             libxslt-dev \
             mariadb-dev \
             pkgconfig \
-            python-dev \
+            python3-dev \
-            py-cffi && \
+            py-cffi \
+            wget && \
     rm -rf /root/* && \
+    rm -rf /tmp/* && \
     rm -rf /var/cache/apk/*
 
-# Run supervisor upon container start
+# Start conpot
-CMD ["/usr/bin/conpot", "--template", "kamstrup_382", "--logfile", "/var/log/conpot/conpot.log", "--config", "/etc/conpot/conpot.cfg"]
+STOPSIGNAL SIGINT
+USER conpot:conpot
+CMD exec /usr/bin/conpot --temp_dir $CONPOT_TMP --template $CONPOT_TEMPLATE --logfile $CONPOT_LOG --config $CONPOT_CONFIG

docker/conpot/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/conpot:1710.svg)](https://microbadger.com/images/dtagdevsec/conpot:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/conpot:1710.svg)](https://microbadger.com/images/dtagdevsec/conpot:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/conpot:1903.svg)](https://microbadger.com/images/dtagdevsec/conpot:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/conpot:1903.svg)](https://microbadger.com/images/dtagdevsec/conpot:1903 "Get your own image badge on microbadger.com")
 
 # conpot
 
@@ -12,4 +12,4 @@ The `docker-compose.yml` contains the necessary settings to test conpot using `d
 
 # ConPot Dashboard
 
-![ConPot Dashboard](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/docker/conpot/doc/dashboard.png)
+![ConPot Dashboard](doc/dashboard.png)

docker/conpot/dist/conpot.cfg

@@ -1,6 +1,10 @@
 [common]
 sensorid = conpot
 
+[virtual_file_system]
+data_fs_url = %(CONPOT_TMP)s
+fs_url = tar:///usr/lib/python3.6/site-packages/conpot/data.tar
+
 [session]
 timeout = 30
 
@@ -10,7 +14,7 @@ group = conpot
 
 [json]
 enabled = True
-filename = /var/log/conpot/conpot.json
+filename = %(CONPOT_JSON_LOG)s
 
 [sqlite]
 enabled = False

docker/conpot/dist/requirements.txt

@@ -1,23 +0,0 @@
-gevent>=1.0
-pysnmp==4.3.5
-pysmi==0.1.3
-lxml
-bottle
-jinja2
-beautifulsoup4
-requests
-sphinx==1.5.5
-libtaxii>=1.1.0
-MySQL-python
-xlrd
-crc16
-enum
-hpfeeds
-modbus-tk
-stix-validator
-stix
-cybox
-bacpypes==0.13.8
-pyghmi
-mixbox
-modbus-tk

docker/conpot/dist/templates/IEC104/template.xml

@@ -0,0 +1,675 @@
+<!-- Copyright (C) 2017  Patrick Reichenberger (University of Passau) <patrick.reichenberger@t-online.de>
+
+ This program is free software; you can redistribute it and/or
+ modify it under the terms of the GNU General Public License
+ as published by the Free Software Foundation; either version 2
+ of the License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc.,
+ 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+-->
+
+<core>
+    <template>
+        <!-- General information about the template -->
+        <entity name="unit">S7-300</entity>
+        <entity name="vendor">Siemens</entity>
+        <entity name="description">Creates a simple device for IEC 60870-5-104</entity>
+        <entity name="protocols">IEC104, SNMP</entity>
+        <entity name="creator">Patrick Reichenberger</entity>
+    </template>
+    <databus>
+        <!-- Core value that can be retrieved from the databus by key -->
+        <key_value_mappings>
+            <!-- SNMPv2-MIB -->
+            <key name="SystemDescription">
+                <value type="value">"Siemens, SIMATIC, S7-300"</value>
+            </key>
+            <key name="sysObjectID">
+                <value type="value">"0.0"</value>
+            </key>
+            <key name="Uptime">
+                <value type="function">conpot.emulators.misc.uptime.Uptime</value>
+            </key>
+            <key name="sysContact">
+                <value type="value">"Corporate IT"</value>
+            </key>
+            <key name="sysName">
+                <value type="value">"DE-BER01"</value>
+            </key>
+            <key name="sysLocation">
+                <value type="value">"BER01, T2E"</value>
+            </key>
+            <key name="sysServices">
+                <value type="value">"72"</value>
+            </key>
+            <!-- IF-MIB -->
+            <key name="ifNumber">
+                <value type="value">1</value>
+            </key>
+            <key name="ifIndex">
+                <value type="value">1</value>
+            </key>
+            <key name="ifDescr">
+                <value type="value">"Siemens, SIMATIC NET, CP 343-1 PN, 6GK7 343-1EX21-0XE0, HW: Version 2, FW: Version V1.2.3, Ethernet Port 1, Rack 0, 100Mbit"</value>
+            </key>
+            <key name="ifType">
+                <value type="value">6</value>
+            </key>
+            <key name="ifMtu">
+                <value type="value">1000</value>
+            </key>
+            <key name="ifSpeed">
+                <value type="value">100000000</value>
+            </key>
+            <key name="ifPhysAddress">
+                <value type="value">"\x00\x0e\x8c\x29\xc5\x1a"</value>
+            </key>
+            <key name="ifAdminStatus">
+                <value type="value">1</value>
+            </key>
+            <key name="ifOperStatus">
+                <value type="value">1</value>
+            </key>
+            <key name="ifLastChange">
+                <value type="function">conpot.emulators.misc.uptime.Uptime</value>
+            </key>
+            <key name="FacilityName">
+                <value type="value">"Compagnie Generale des Eaux"</value>
+            </key>
+            <key name="0">
+                <value type="value">0</value>
+            </key>
+            <key name="1">
+                <value type="value">1</value>
+            </key>
+            <key name="ifInOctets">
+                <value type="value">1618895</value>
+            </key>
+            <key name="ifInUcastPkts">
+                <value type="value">7018</value>
+            </key>
+            <key name="ifInNUcastPkts">
+                <value type="value">291</value>
+            </key>
+            <key name="ifOutOctets">
+                <value type="value">455107</value>
+            </key>
+            <key name="ifOutUcastPkts">
+                <value type="value">872264</value>
+            </key>
+            <key name="ifOutUNcastPkts">
+                <value type="value">143</value>
+            </key>
+
+            <!-- IP-MIB -->
+            <key name="ipForwarding">
+                <value type="value">2</value>
+            </key>
+            <key name="ipDefaultTTL">
+                <value type="value">60</value>
+            </key>
+            <key name="ipInReceives">
+                <value type="value">31271</value>
+            </key>
+            <key name="ipInHdrErrors">
+                <value type="value">0</value>
+            </key>
+            <key name="ipInAddrErrors">
+                <value type="value">0</value>
+            </key>
+            <key name="ipForwDatagrams">
+                <value type="value">0</value>
+            </key>
+            <key name="ipInUnknownProtos">
+                <value type="value">0</value>
+            </key>
+            <key name="ipInDiscards">
+                <value type="value">0</value>
+            </key>
+            <key name="ipInDelivers">
+                <value type="value">31282</value>
+            </key>
+            <key name="ipOutRequests">
+                <value type="value">69023</value>
+            </key>
+            <key name="ipOutDiscards">
+                <value type="value">0</value>
+            </key>
+            <key name="ipOutNoRoutes">
+                <value type="value">0</value>
+            </key>
+            <key name="ipReasmTimeout">
+                <value type="value">60</value>
+            </key>
+            <key name="ipReasmReqds">
+                <value type="value">7</value>
+            </key>
+            <key name="ipReasmOKs">
+                <value type="value">3</value>
+            </key>
+            <key name="ipReasmFails">
+                <value type="value">0</value>
+            </key>
+            <key name="ipFragOKs">
+                <value type="value">0</value>
+            </key>
+            <key name="ipFragFails">
+                <value type="value">0</value>
+            </key>
+            <key name="ipFragCreates">
+                <value type="value">0</value>
+            </key>
+            <key name="ipAdEntAddr">
+                <value type="value">"217.172.190.137"</value>
+            </key>
+            <key name="ipAdEntIfIndex">
+                <value type="value">1</value>
+            </key>
+            <key name="ipAdEntNetMask">
+                <value type="value">"255.255.255.255"</value>
+            </key>
+            <key name="ipAdEntBcastAddr">
+                <value type="value">1</value>
+            </key>
+            <key name="ipAdEntReasmMaxSize">
+                <value type="value">65528</value>
+            </key>
+            <key name="ipRoutingDiscards">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInMsgs">
+                <value type="value">4</value>
+            </key>
+            <key name="icmpInErrors">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInDestUnreachs">
+                <value type="value">1</value>
+            </key>
+            <key name="icmpInTimeExcds">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInParmProbs">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInSrcQuenchs">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInRedirects">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInEchos">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInEchoReps">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInTimestamps">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInTimestampReps">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInAddrMasks">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInAddrMaskReps">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutMsgs">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutErrors">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutDestUnreachs">
+                <value type="value">144</value>
+            </key>
+            <key name="icmpOutTimeExcds">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutParmProbs">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutSrcQuenchs">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutRedirects">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutEchos">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutEchoReps">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutTimestamps">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutTimestampReps">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutAddrMasks">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutAddrMaskReps">
+                <value type="value">0</value>
+            </key>
+
+            <!-- TCP-MIB -->
+            <key name="tcpRtoAlgorithm">
+                <value type="value">2</value>
+            </key>
+            <key name="tcpRtoMin">
+                <value type="value">0</value>
+            </key>
+            <key name="tcpRtoMax">
+                <value type="value">100</value>
+            </key>
+            <key name="tcpMaxConn">
+                <value type="value">-1</value>
+            </key>
+            <key name="tcpActiveOpens">
+                <value type="value">0</value>
+            </key>
+            <key name="tcpPassiveOpens">
+                <value type="value">101</value>
+            </key>
+            <key name="tcpAttemptFails">
+                <value type="value">42</value>
+            </key>
+            <key name="tcpEstabResets">
+                <value type="value">45</value>
+            </key>
+            <key name="tcpCurrEstab">
+                <value type="value">0</value>
+            </key>
+            <key name="tcpInSegs">
+                <value type="value">30321</value>
+            </key>
+            <key name="tcpOutSegs">
+                <value type="value">67821</value>
+            </key>
+            <key name="tcpRetransSegs">
+                <value type="value">2511</value>
+            </key>
+            <key name="tcpConnState">
+                <value type="value">2</value>
+            </key>
+            <key name="tcpConnLocalAddress">
+                <value type="value">"217.172.190.137"</value>
+            </key>
+            <key name="tcpConnLocalPort">
+                <value type="value">2404</value>
+            </key>
+            <key name="tcpConnRemAddress">
+                <value type="value">"0.0.0.0"</value>
+            </key>
+            <key name="tcpConnRemPort">
+                <value type="value">0</value>
+            </key>
+            <key name="tcpInErrs">
+                <value type="value">1</value>
+            </key>
+            <key name="tcpOutRsts">
+                <value type="value">728</value>
+            </key>
+            <!-- UDP-MIB -->
+            <key name="udpInDatagrams">
+                <value type="value">1441</value>
+            </key>
+            <key name="udpNoPorts">
+                <value type="value">1280</value>
+            </key>
+            <key name="udpInErrors">
+                <value type="value">23</value>
+            </key>
+            <key name="udpOutDatagrams">
+                <value type="value">47</value>
+            </key>
+            <key name="udpLocalAddress">
+                <value type="value">"217.172.190.137"</value>
+            </key>
+            <key name="udpLocalPort">
+                <value type="value">161</value>
+            </key>
+            <key name="SystemName">
+                <value type="value">"CP 343-1 IT"</value>
+            </key>
+
+
+            <!-- IEC104 Protocol parameter -->
+            <!-- Timeout of connection establishment -->
+            <key name="T_0">
+                <value type="value">30</value>
+            </key>
+            <!-- Timeout of send or test APDUs (Wartezeit auf Quittung) -->
+            <key name="T_1">
+                <value type="value">15</value>
+            </key>
+            <!-- Timeout for acknowledges in case of no data messages T_2 < T_1 (Quittieren nach x sek) -->
+            <key name="T_2">
+                <value type="value">10</value>
+            </key>
+            <!-- Timeout for sending test frames in case of a long idle state -->
+            <key name="T_3">
+                <value type="value">20</value>
+            </key>
+            <!-- Maximum difference receive sequence number to send state variable (Max. Anzahl unquittierter Telegramme) -->
+            <!-- not implemented yet -->
+            <key name="k">
+                <value type="value">12</value>
+            </key>
+            <!-- Latest acknowledge after receiving w I-format APDUs (Quittieren nach w Telegrammen) -->
+            <key name="w">
+                <value type="value">8</value>
+            </key>
+            <!-- Maximum frame size (in bytes) -->
+            <key name="MaxFrameSize">
+                <value type="value">254</value>
+            </key>
+
+            <!-- Devices -->
+            <!-- 13- -->
+            <key name="13_20">
+                <value type="value">1</value>
+            </key>
+            <key name="13_21">
+                <value type="value">0</value>
+            </key>
+            <key name="13_22">
+                <value type="value">0</value>
+            </key>
+            <key name="13_24">
+                <value type="value">1</value>
+            </key>
+            <key name="13_25">
+                <value type="value">1</value>
+            </key>
+            <key name="13_32">
+                <value type="value">1</value>
+            </key>
+            <key name="13_33">
+                <value type="value">1</value>
+            </key>
+            <key name="13_34">
+                <value type="value">1</value>
+            </key>
+            <key name="13_35">
+                <value type="value">1</value>
+            </key>
+            <key name="13_36">
+                <value type="value">1</value>
+            </key>
+            <key name="13_37">
+                <value type="value">1</value>
+            </key>
+            <key name="13_38">
+                <value type="value">1</value>
+            </key>
+            <key name="13_39">
+                <value type="value">1</value>
+            </key>
+            <key name="13_40">
+                <value type="value">0</value>
+            </key>
+            <key name="13_41">
+                <value type="value">1</value>
+            </key>
+            <key name="13_42">
+                <value type="value">0</value>
+            </key>
+
+            <!-- 22- -->
+            <key name="22_19">
+                <value type="value">1</value>
+            </key>
+            <key name="22_20">
+                <value type="value">1</value>
+            </key>
+            <key name="22_21">
+                <value type="value">0</value>
+            </key>
+            <key name="22_22">
+                <value type="value">0</value>
+            </key>
+            <key name="22_24">
+                <value type="value">1</value>
+            </key>
+            <key name="22_25">
+                <value type="value">1</value>
+            </key>
+            <key name="22_42">
+                <value type="value">1</value>
+            </key>
+            <key name="22_43">
+                <value type="value">1</value>
+            </key>
+            <key name="22_54">
+                <value type="value">1</value>
+            </key>
+
+            <!-- 33- -->
+            <key name="33_2">
+                <value type="value">1</value>
+            </key>
+            <key name="33_3">
+                <value type="value">2</value>
+            </key>
+            <key name="33_4">
+                <value type="value">1</value>
+            </key>
+            <key name="33_5">
+                <value type="value">2</value>
+            </key>
+            <key name="33_6">
+                <value type="value">2</value>
+            </key>
+            <key name="33_7">
+                <value type="value">1</value>
+            </key>
+            <key name="33_8">
+                <value type="value">1</value>
+            </key>
+            <key name="33_9">
+                <value type="value">1</value>
+            </key>
+            <key name="33_10">
+                <value type="value">1</value>
+            </key>
+            <key name="33_11">
+                <value type="value">1</value>
+            </key>
+
+            <!-- 60- -->
+            <key name="60_6">
+                <value type="value">2</value>
+            </key>
+            <key name="60_7">
+                <value type="value">1</value>
+            </key>
+            <key name="60_8">
+                <value type="value">1</value>
+            </key>
+            <key name="60_9">
+                <value type="value">1</value>
+            </key>
+            <key name="60_20">
+                <value type="value">1</value>
+            </key>
+            <key name="60_21">
+                <value type="value">1</value>
+            </key>
+            <key name="60_32">
+                <value type="value">1</value>
+            </key>
+            <key name="60_34">
+                <value type="value">1</value>
+            </key>
+            <key name="60_35">
+                <value type="value">1</value>
+            </key>
+            <key name="60_36">
+                <value type="value">1</value>
+            </key>
+
+            <!-- 100- -->
+            <key name="100_12">
+                <value type="value">103</value>
+            </key>
+            <key name="100_13">
+                <value type="value">31</value>
+            </key>
+            <key name="100_51">
+                <value type="value">-49</value>
+            </key>
+            <key name="100_108">
+                <value type="value">28871</value>
+            </key>
+            <key name="100_109">
+                <value type="value">13781</value>
+            </key>
+            <key name="100_178">
+                <value type="value">119</value>
+            </key>
+            <key name="100_179">
+                <value type="value">219</value>
+            </key>
+            <key name="100_190">
+                <value type="value">1009</value>
+            </key>
+            <key name="100_191">
+                <value type="value">-2</value>
+            </key>
+            <key name="100_192">
+                <value type="value">701</value>
+            </key>
+            <key name="100_193">
+                <value type="value">441</value>
+            </key>
+
+            <!-- 101- -->
+            <key name="101_63">
+                <value type="value">103</value>
+            </key>
+            <key name="101_205">
+                <value type="value">31</value>
+            </key>
+            <key name="101_100">
+                <value type="value">5</value>
+            </key>
+            <key name="101_101">
+                <value type="value">49</value>
+            </key>
+            <key name="101_102">
+                <value type="value">119</value>
+            </key>
+            <key name="101_105">
+                <value type="value">500</value>
+            </key>
+            <key name="101_106">
+                <value type="value">1</value>
+            </key>
+
+            <!-- 107- -->
+            <key name="107_3">
+                <value type="value">16.2</value>
+            </key>
+            <key name="107_77">
+                <value type="value">15.9</value>
+            </key>
+            <key name="107_78">
+                <value type="value">512.1</value>
+            </key>
+            <key name="107_79">
+                <value type="value">433.4</value>
+            </key>
+            <key name="107_90">
+                <value type="value">344.4</value>
+            </key>
+            <key name="107_130">
+                <value type="value">-0.44013</value>
+            </key>
+            <key name="107_131">
+                <value type="value">43.0</value>
+            </key>
+            <key name="107_132">
+                <value type="value">41.2</value>
+            </key>
+            <key name="107_141">
+                <value type="value">12.1</value>
+            </key>
+            <key name="107_200">
+                <value type="value">91</value>
+            </key>
+            <key name="107_201">
+                <value type="value">98.8</value>
+            </key>
+            <key name="107_202">
+                <value type="value">110</value>
+            </key>
+            <key name="107_203">
+                <value type="value">85.1</value>
+            </key>
+            <key name="107_204">
+                <value type="value">85.2</value>
+            </key>
+            <key name="107_205">
+                <value type="value">410</value>
+            </key>
+            <key name="107_206">
+                <value type="value">592</value>
+            </key>
+            <key name="107_207">
+                <value type="value">1.5</value>
+            </key>
+            <key name="107_208">
+                <value type="value">44.7</value>
+            </key>
+            <key name="107_209">
+                <value type="value">11.9</value>
+            </key>
+            <key name="107_210">
+                <value type="value">221.45</value>
+            </key>
+            <key name="107_211">
+                <value type="value">13.4</value>
+            </key>
+            <key name="107_212">
+                <value type="value">0.000402</value>
+            </key>
+
+            <!-- 109- -->
+            <key name="109_3">
+                <value type="value">16.2</value>
+            </key>
+            <key name="109_7">
+                <value type="value">15.9</value>
+            </key>
+            <key name="109_8">
+                <value type="value">880</value>
+            </key>
+            <key name="109_10">
+                <value type="value">344.4</value>
+            </key>
+            <key name="109_40">
+                <value type="value">41.2</value>
+            </key>
+            <key name="109_41">
+                <value type="value">12.1</value>
+            </key>
+
+            <key name="empty">
+                <value type="value">""</value>
+            </key>
+        </key_value_mappings>
+    </databus>
+</core>

docker/conpot/dist/templates/default/template.xml

@@ -0,0 +1,78 @@
+<core>
+    <template>
+        <!-- General information about the template -->
+        <entity name="unit">S7-200</entity>
+        <entity name="vendor">Siemens</entity>
+        <entity name="description">Rough simulation of a basic Siemens S7-200 CPU with 2 slaves</entity>
+        <entity name="protocols">HTTP, MODBUS, s7comm, SNMP</entity>
+        <entity name="creator">the conpot team</entity>
+    </template>
+    <databus>
+        <!-- Core value that can be retrieved from the databus by key -->
+        <key_value_mappings>
+            <key name="FacilityName">
+                <value type="value">"DoE Water Service"</value>
+            </key>
+            <key name="SystemName">
+                <value type="value">"Central Pump"</value>
+            </key>
+            <key name="SystemDescription">
+                <value type="value">"Pump Control Unit"</value>
+            </key>
+            <key name="Uptime">
+                <value type="function">conpot.emulators.misc.uptime.Uptime</value>
+            </key>
+            <key name="sysObjectID">
+                <value type="value">"0.0"</value>
+            </key>
+            <key name="sysContact">
+                <value type="value">"DoE"</value>
+            </key>
+            <key name="sysName">
+                <value type="value">"Pump Control Unit"</value>
+            </key>
+            <key name="sysLocation">
+                <value type="value">"DoE"</value>
+            </key>
+            <key name="sysServices">
+                <value type="value">"72"</value>
+            </key>
+            <key name="memoryModbusSlave0BlockA">
+                <value type="value">[random.randint(0,1) for b in range(0,128)]</value>
+            </key>
+            <key name="memoryModbusSlave0BlockB">
+                <value type="value">[random.randint(0,1) for b in range(0,32)]</value>
+            </key>
+            <key name="memoryModbusSlave255BlockA">
+                <value type="value">[random.randint(0,1) for b in range(0,128)]</value>
+            </key>
+            <key name="memoryModbusSlave255BlockB">
+                <value type="value">[random.randint(0,1) for b in range(0,32)]</value>
+            </key>
+            <key name="memoryModbusSlave1BlockA">
+                <value type="value">[random.randint(0,1) for b in range(0,128)]</value>
+            </key>
+            <key name="memoryModbusSlave1BlockB">
+                <value type="value">[random.randint(0,1) for b in range(0,32)]</value>
+            </key>
+            <key name="memoryModbusSlave2BlockC">
+                <value type="value">[random.randint(0,1) for b in range(0,8)]</value>
+            </key>
+            <key name="memoryModbusSlave2BlockD">
+                <value type="value">[0 for b in range(0,32)]</value>
+            </key>
+            <key name="Copyright">
+                <value type="value">"Original Siemens Equipment"</value>
+            </key>
+            <key name="s7_id">
+                <value type="value">"88111222"</value>
+            </key>
+            <key name="s7_module_type">
+                <value type="value">"IM151-8 PN/DP CPU"</value>
+            </key>
+            <key name="empty">
+                <value type="value">""</value>
+            </key>
+        </key_value_mappings>
+    </databus>
+</core>

docker/conpot/dist/templates/guardian_ast/template.xml

@@ -0,0 +1,93 @@
+<core>
+    <template>
+        <!-- General information about the template -->
+        <entity name="unit">Guardian AST tank-monitoring system</entity>
+        <entity name="vendor">Guardian</entity>
+        <entity name="description">Guardian AST tank-monitoring system</entity>
+        <entity name="protocols">guardian_ast</entity>
+        <entity name="creator">the conpot team</entity>
+    </template>
+    <databus>
+        <!-- Core value that can be retrieved from the databus by key -->
+        <key_value_mappings>
+            <key name="product1">
+                <value type="value">"SUPER"</value>
+            </key>
+            <key name="product2">
+                <value type="value">"UNLEAD"</value>
+            </key>
+            <key name="product3">
+                <value type="value">"DIESEL"</value>
+            </key>
+            <key name="product4">
+                <value type="value">"ADBLUE"</value>
+            </key>
+            <key name="station_name">
+                <value type="value">"AVIA"</value>
+            </key>
+            <key name="vol1">
+                <value type="value">random.randint(1000, 9050)</value>
+            </key>
+            <key name="vol2">
+                <value type="value">random.randint(1000, 9050)</value>
+            </key>
+            <key name="vol3">
+                <value type="value">random.randint(1000, 9050)</value>
+            </key>
+            <key name="vol4">
+                <value type="value">random.randint(1000, 9050)</value>
+            </key>
+            <key name="ullage1">
+                <value type="value">random.randint(3000, 9999)</value>
+            </key>
+            <key name="ullage2">
+                <value type="value">random.randint(3000, 9999)</value>
+            </key>
+            <key name="ullage3">
+                <value type="value">random.randint(3000, 9999)</value>
+            </key>
+            <key name="ullage4">
+                <value type="value">random.randint(3000, 9999)</value>
+            </key>
+            <key name="height1">
+                <value type="value">round(random.uniform(25.00, 75.99), 2)</value>
+            </key>
+            <key name="height2">
+                <value type="value">round(random.uniform(25.00, 75.99), 2)</value>
+            </key>
+            <key name="height3">
+                <value type="value">round(random.uniform(25.00, 75.99), 2)</value>
+            </key>
+            <key name="height4">
+                <value type="value">round(random.uniform(25.00, 75.99), 2)</value>
+            </key>
+            <key name="h2o1">
+                <value type="value">round(random.uniform(0.0, 9.99), 2)</value>
+            </key>
+            <key name="h2o2">
+                <value type="value">round(random.uniform(0.0, 9.99), 2)</value>
+            </key>
+            <key name="h2o3">
+                <value type="value">round(random.uniform(0.0, 9.99), 2)</value>
+            </key>
+            <key name="h2o4">
+                <value type="value">round(random.uniform(0.0, 9.99), 2)</value>
+            </key>
+            <key name="temp1">
+                <value type="value">round(random.uniform(50.0, 59.99), 2)</value>
+            </key>
+            <key name="temp2">
+                <value type="value">round(random.uniform(50.0, 59.99), 2)</value>
+            </key>
+            <key name="temp3">
+                <value type="value">round(random.uniform(50.0, 59.99), 2)</value>
+            </key>
+            <key name="temp4">
+                <value type="value">round(random.uniform(50.0, 59.99), 2)</value>
+            </key>
+            <key name="empty">
+                <value type="value">""</value>
+            </key>
+        </key_value_mappings>
+    </databus>
+</core>

docker/conpot/dist/templates/ipmi/template.xml

@@ -0,0 +1,18 @@
+<core>
+    <template>
+        <!-- General information about the template -->
+        <entity name="unit">371</entity>
+        <entity name="vendor">IPMI</entity>
+        <entity name="description">Creates a simple IPMI device</entity>
+        <entity name="protocols">IPMI</entity>
+        <entity name="creator">Lukas Rist</entity>
+    </template>
+    <databus>
+        <!-- Core value that can be retrieved from the databus by key -->
+        <key_value_mappings>
+          <key name="SystemName">
+              <value type="value">"DoE"</value>
+          </key>
+        </key_value_mappings>
+    </databus>
+</core>

docker/conpot/dist/templates/kamstrup_382/template.xml

@@ -413,43 +413,43 @@
                 <value type="value">''</value>
             </key>
             <key name="nameserver_1">
-                <value type="value">'192.168.254.111'</value>
+                <value type="value">'0.0.0.0'</value>
             </key>
             <key name="nameserver_2">
-                <value type="value">'192.168.254.112'</value>
+                <value type="value">'0.0.0.0'</value>
             </key>
             <key name="nameserver_3">
                 <value type="value">'0.0.0.0'</value>
             </key>
             <key name="mac_address">
-                <value type="value">'00:13:EA:00:72:FA'</value>
+                <value type="value">'00:13:EA:00:00:00'</value>
             </key>
             <key name="use_dhcp">
                 <value type="value">'YES'</value>
             </key>
             <key name="ip_addr">
-                <value type="value">'192.168.201.101'</value>
+                <value type="value">'192.168.1.210'</value>
             </key>
             <key name="ip_gateway">
-                <value type="value">'192.168.201.254'</value>
+                <value type="value">'192.168.1.1'</value>
             </key>
             <key name="ip_subnet">
                 <value type="value">'255.255.255.0'</value>
             </key>
             <key name="ip_addr_dhcp">
-                <value type="value">'192.168.200.1'</value>
+                <value type="value">'192.168.0.1'</value>
             </key>
             <key name="ip_gateway_dhcp">
-                <value type="value">'192.168.200.254'</value>
+                <value type="value">'192.168.0.254'</value>
             </key>
             <key name="ip_subnet_dhcp">
                 <value type="value">'255.255.255.0'</value>
             </key>
             <key name="kap_a_server_hostname">
-                <value type="value">'de_fra_lxg00.local.dom'</value>
+                <value type="value">'pwr_ctrl_mgmt01.int.local'</value>
             </key>
             <key name="kap_a_server_ip">
-                <value type="value">'192.168.254.201'</value>
+                <value type="value">'10.232.15.242'</value>
             </key>
             <key name="kap_a_server_port">
                 <value type="value">'50'</value>

docker/conpot/dist/templates/proxy/template.xml

@@ -0,0 +1,15 @@
+    <core>
+    <template>
+        <!-- General information about the template -->
+        <entity name="unit">Proxy</entity>
+        <entity name="vendor">None</entity>
+        <entity name="description">Sample template that demonstrates the proxy feature.</entity>
+        <entity name="protocols">Proxy</entity>
+        <entity name="creator">the conpot team</entity>
+    </template>
+    <databus>
+        <!-- Core value that can be retrieved from the databus by key -->
+        <key_value_mappings>
+        </key_value_mappings>
+    </databus>
+</core>

docker/conpot/docker-compose.yml

@@ -1,19 +1,131 @@
-version: '2.1'
+# CONPOT TEMPLATE=[default, IEC104, guardian_ast, ipmi, kamstrup_382, proxy]
+version: '2.3'
 
 networks:
-  conpot_local:
+  conpot_local_default:
+  conpot_local_IEC104:
+  conpot_local_guardian_ast:
+  conpot_local_ipmi:
+  conpot_local_kamstrup_382:
 
 services:
 
-# Conpot service
+# Conpot default service
-  conpot:
+  conpot_default:
-    container_name: conpot
+    build: .
+    container_name: conpot_default
     restart: always
+    environment:
+     - CONPOT_CONFIG=/etc/conpot/conpot.cfg
+     - CONPOT_JSON_LOG=/var/log/conpot/conpot_default.json
+     - CONPOT_LOG=/var/log/conpot/conpot_default.log
+     - CONPOT_TEMPLATE=default
+     - CONPOT_TMP=/tmp/conpot
+    tmpfs:
+     - /tmp/conpot:uid=2000,gid=2000
     networks:
-     - conpot_local
+     - conpot_local_default
+    ports:
+#    - "69:69"        
+     - "80:80"
+     - "102:102"
+     - "161:161"
+     - "502:502"
+#     - "623:623"
+     - "2121:21"
+     - "44818:44818"
+     - "47808:47808"
+    image: "dtagdevsec/conpot:1903"
+    read_only: true
+    volumes:
+     - /data/conpot/log:/var/log/conpot
+
+# Conpot IEC104 service
+  conpot_IEC104:
+    build: .
+    container_name: conpot_IEC104
+    restart: always
+    environment:
+     - CONPOT_CONFIG=/etc/conpot/conpot.cfg
+     - CONPOT_JSON_LOG=/var/log/conpot/conpot_IEC104.json
+     - CONPOT_LOG=/var/log/conpot/conpot_IEC104.log
+     - CONPOT_TEMPLATE=IEC104
+     - CONPOT_TMP=/tmp/conpot
+    tmpfs:
+     - /tmp/conpot:uid=2000,gid=2000
+    networks:
+     - conpot_local_IEC104
+    ports:
+#     - "161:161"
+     - "2404:2404"
+    image: "dtagdevsec/conpot:1903"
+    read_only: true
+    volumes:
+     - /data/conpot/log:/var/log/conpot
+
+# Conpot guardian_ast service
+  conpot_guardian_ast:
+    build: .
+    container_name: conpot_guardian_ast
+    restart: always
+    environment:
+     - CONPOT_CONFIG=/etc/conpot/conpot.cfg
+     - CONPOT_JSON_LOG=/var/log/conpot/conpot_guardian_ast.json
+     - CONPOT_LOG=/var/log/conpot/conpot_guardian_ast.log
+     - CONPOT_TEMPLATE=guardian_ast
+     - CONPOT_TMP=/tmp/conpot
+    tmpfs:
+     - /tmp/conpot:uid=2000,gid=2000
+    networks:
+     - conpot_local_guardian_ast
+    ports:
+     - "10001:10001"
+    image: "dtagdevsec/conpot:1903"
+    read_only: true
+    volumes:
+     - /data/conpot/log:/var/log/conpot
+
+# Conpot ipmi
+  conpot_ipmi:
+    build: .
+    container_name: conpot_ipmi
+    restart: always
+    environment:
+     - CONPOT_CONFIG=/etc/conpot/conpot.cfg
+     - CONPOT_JSON_LOG=/var/log/conpot/conpot_ipmi.json
+     - CONPOT_LOG=/var/log/conpot/conpot_ipmi.log
+     - CONPOT_TEMPLATE=ipmi
+     - CONPOT_TMP=/tmp/conpot
+    tmpfs:
+     - /tmp/conpot:uid=2000,gid=2000
+    networks:
+     - conpot_local_ipmi
+    ports:
+     - "623:623"
+    image: "dtagdevsec/conpot:1903"
+    read_only: true
+    volumes:
+     - /data/conpot/log:/var/log/conpot
+
+# Conpot kamstrup_382
+  conpot_kamstrup_382:
+    build: .
+    container_name: conpot_kamstrup_382
+    restart: always
+    environment:
+     - CONPOT_CONFIG=/etc/conpot/conpot.cfg
+     - CONPOT_JSON_LOG=/var/log/conpot/conpot_kamstrup_382.json
+     - CONPOT_LOG=/var/log/conpot/conpot_kamstrup_382.log
+     - CONPOT_TEMPLATE=kamstrup_382
+     - CONPOT_TMP=/tmp/conpot
+    tmpfs:
+     - /tmp/conpot:uid=2000,gid=2000
+    networks:
+     - conpot_local_kamstrup_382
     ports:
      - "1025:1025"
      - "50100:50100"
-    image: "dtagdevsec/conpot:1710"
+    image: "dtagdevsec/conpot:1903"
+    read_only: true
     volumes:
      - /data/conpot/log:/var/log/conpot

docker/cowrie/Dockerfile

@@ -1,35 +1,70 @@
 FROM alpine
-MAINTAINER MO
 
 # Include dist
 ADD dist/ /root/dist/
 
 # Get and install dependencies & packages
-RUN apk -U upgrade && \
+RUN apk -U --no-cache add \
-    apk add git procps py-pip mpfr-dev openssl-dev mpc1-dev libffi-dev build-base python python-dev py-mysqldb py-setuptools gmp-dev && \
+                       bash \
+                       build-base \
+                       git \
+                       gmp-dev \
+                       libcap \
+                       libffi-dev \
+                       mpc1-dev \
+                       mpfr-dev \
+                       openssl \
+                       openssl-dev \
+                       python \
+                       python-dev \
+                       py-bcrypt \
+                       py-mysqldb \
+                       py-pip \
+                       py-requests \
+                       py-setuptools && \
 
 # Setup user
     addgroup -g 2000 cowrie && \
-    adduser -S -s /bin/bash -u 2000 -D -g 2000 cowrie && \
+    adduser -S -s /bin/ash -u 2000 -D -g 2000 cowrie && \
 
-# Install cowrie from git
+# Install cowrie
-    git clone https://github.com/micheloosterhof/cowrie.git /home/cowrie/cowrie/ && \
+    mkdir -p /home/cowrie && \
-    cd /home/cowrie/cowrie && \
+    cd /home/cowrie && \
-    pip install --no-cache-dir --upgrade cffi && \
+    git clone --depth=1 https://github.com/micheloosterhof/cowrie -b 1.5.3 && \
-    pip install --no-cache-dir -U -r requirements.txt && \
+    cd cowrie && \
+    mkdir -p log && \
+    pip install --upgrade pip && \
+    pip install --upgrade -r requirements.txt && \
 
-# Setup user, groups and configs
+# Setup configs
+    setcap cap_net_bind_service=+ep /usr/bin/python2.7 && \
     cp /root/dist/cowrie.cfg /home/cowrie/cowrie/cowrie.cfg && \
-    cp /root/dist/userdb.txt /home/cowrie/cowrie/data/userdb.txt && \
+    chown cowrie:cowrie -R /home/cowrie/* /usr/lib/python2.7/site-packages/twisted/plugins && \
-    chown cowrie:cowrie -R /home/cowrie/* && \
+
+# Start Cowrie once to prevent dropin.cache errors upon container start caused by read-only filesystem
+    su - cowrie -c "export PYTHONPATH=/home/cowrie/cowrie:/home/cowrie/cowrie/src && \
+                    cd /home/cowrie/cowrie && \
+                    /usr/bin/twistd --uid=2000 --gid=2000 -y cowrie.tac --pidfile cowrie.pid cowrie &" && \
+    sleep 10 && \
 
 # Clean up
+    apk del --purge build-base \
+                    git \
+                    gmp-dev \
+                    libcap \
+                    libffi-dev \
+                    mpc1-dev \
+                    mpfr-dev \
+                    openssl-dev \
+                    python-dev \
+                    py-mysqldb \
+                    py-pip && \
     rm -rf /root/* && \
-    apk del git py-pip mpfr-dev mpc1-dev libffi-dev build-base py-mysqldb gmp-dev python-dev && \
+    rm -rf /var/cache/apk/* && \
-    rm -rf /var/cache/apk/*
+    rm -rf /home/cowrie/cowrie/cowrie.pid
 
 # Start cowrie
-ENV PYTHONPATH /home/cowrie/cowrie
+ENV PYTHONPATH /home/cowrie/cowrie:/home/cowrie/cowrie/src
 WORKDIR /home/cowrie/cowrie
-USER cowrie
+USER cowrie:cowrie
-CMD ["/usr/bin/twistd", "--nodaemon", "-y", "cowrie.tac", "--pidfile", "var/run/cowrie.pid", "cowrie"]
+CMD ["/usr/bin/twistd", "--nodaemon", "-y", "cowrie.tac", "--pidfile", "/tmp/cowrie/cowrie.pid", "cowrie"]

docker/cowrie/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/cowrie:1710.svg)](https://microbadger.com/images/dtagdevsec/cowrie:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/cowrie:1710.svg)](https://microbadger.com/images/dtagdevsec/cowrie:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/cowrie:1903.svg)](https://microbadger.com/images/dtagdevsec/cowrie:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/cowrie:1903.svg)](https://microbadger.com/images/dtagdevsec/cowrie:1903 "Get your own image badge on microbadger.com")
 
 # cowrie
 
@@ -12,4 +12,4 @@ The `docker-compose.yml` contains the necessary settings to test cowrie using `d
 
 # Cowrie Dashboard
 
-![Cowrie Dashboard](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/docker/cowrie/doc/dashboard.png)
+![Cowrie Dashboard](doc/dashboard.png)

docker/cowrie/dist/cowrie.cfg

@@ -1,472 +1,57 @@
-#
-# Cowrie configuration file (cowrie.cfg)
-#
-
-# ============================================================================
-# General Honeypot Options
-# ============================================================================
 [honeypot]
-
-# Sensor name is used to identify this Cowrie instance. Used by the database
-# logging modules such as mysql.
-#
-# If not specified, the logging modules will instead use the IP address of the
-# server as the sensor name.
-#
-# (default: not specified)
-#sensor_name=t-pot
-
-# Hostname for the honeypot. Displayed by the shell prompt of the virtual
-# environment
-#
-# (default: svr04)
 hostname = ubuntu
-
-
-# Directory where to save log files in.
-#
-# (default: log)
 log_path = log
-
-
-# Directory where to save downloaded artifacts in.
-#
-# (default: dl)
 download_path = dl
-
-
-# Directory for miscellaneous data files, such as the password database.
-#
-# (default: data_path)
-data_path = data
-
-
-# Directory where virtual file contents are kept in.
-#
-# This is only used by commands like 'cat' to display the contents of files.
-# Adding files here is not enough for them to appear in the honeypot - the
-# actual virtual filesystem is kept in filesystem_file (see below)
-#
-# (default: honeyfs)
-contents_path = honeyfs
-
-
-# File in the Python pickle format containing the virtual filesystem.
-#
-# This includes the filenames, paths, permissions for the Cowrie filesystem,
-# but not the file contents. This is created by the bin/createfs utility from
-# a real template linux installation.
-#
-# (default: fs.pickle)
-filesystem_file = data/fs.pickle
-
-
-# Directory for creating simple commands that only output text.
-#
-# The command must be placed under this directory with the proper path, such
-# as:
-#   txtcmds/usr/bin/vi
-# The contents of the file will be the output of the command when run inside
-# the honeypot.
-#
-# In addition to this, the file must exist in the virtual filesystem
-#
-# (default: txtcmds)
-txtcmds_path = txtcmds
-
-
-# Maximum file size (in bytes) for downloaded files to be stored in 'download_path'.
-# A value of 0 means no limit. If the file size is known to be too big from the start,
-# the file will not be stored on disk at all.
-#
-# (default: 0)
-#download_limit_size = 10485760
-
-
-# TTY logging will log a transcript of the complete terminal interaction in UML
-# compatible format.
-# (default: true)
-ttylog = true
-
-
-
-# ============================================================================
-# Network Specific Options
-# ============================================================================
-
-
-# IP address to bind to when opening outgoing connections. Used by wget and
-# curl commands.
-#
-# (default: not specified)
-#out_addr = 0.0.0.0
-
-
-# Fake address displayed as the address of the incoming connection.
-# This doesn't affect logging, and is only used by honeypot commands such as
-# 'w' and 'last'
-#
-# If not specified, the actual IP address is displayed instead (default
-# behaviour).
-#
-# (default: not specified)
-#fake_addr = 192.168.66.254
-
-
-# The IP address on which this machine is reachable on from the internet.
-# Useful if you use portforwarding or other mechanisms. If empty, Cowrie
-# will determine by itself. Used in 'netstat' output
-#
-#internet_facing_ip = 9.9.9.9
-
-
-# Enable to log the public IP of the honeypot (useful if listening on 127.0.0.1)
-# IP address is obtained by querying http://myip.threatstream.com
 report_public_ip = true
-
+share_path= share/cowrie
-
+state_path = /tmp/cowrie/data
-
+etc_path = etc
-# ============================================================================
+contents_path = honeyfs
-# Authentication Specific Options
+txtcmds_path = txtcmds
-# ============================================================================
+ttylog = true
-
+ttylog_path = log/tty
-
+interactive_timeout = 180
-# Class that implements the checklogin() method.
+authentication_timeout = 120
-#
+backend = shell
-# Class must be defined in cowrie/core/auth.py
-# Default is the 'UserDB' class which uses the password database.
-#
-# Alternatively the 'AuthRandom' class can be used, which will let
-# a user login after a random number of attempts.
-# It will also cache username/password combinations that allow login.
-#
-#auth_class = UserDB
-
-# When AuthRandom is used also set the
-#  auth_class_parameters: <min try>, <max try>, <maxcache>
-#  for example: 2, 5, 10 = allows access after randint(2,5) attempts
-#  and cache 10 combinations.
-#
 auth_class = AuthRandom
 auth_class_parameters = 2, 5, 10
-
-
-# No authentication checking at all
-# enabling 'auth_none' will enable the ssh2 'auth_none' authentication method
-# this allows the requested user in without any verification at all
-#
-# (default: false)
-#auth_none_enabled = false
-
-
-
-# ============================================================================
-# Historical SSH Specific Options
-# historical options in [honeypot] that have not yet been moved to [ssh]
-# ============================================================================
-
-# Source Port to report in logs (useful if you use iptables to forward ports to Cowrie)
 reported_ssh_port = 22
+data_path = /tmp/cowrie/data
 
+[shell]
+filesystem = share/cowrie/fs.pickle 
+processes = share/cowrie/cmdoutput.json
+arch = linux-x64-lsb
+kernel_version = 3.2.0-4-amd64
+kernel_build_string = #1 SMP Debian 3.2.68-1+deb7u1
+hardware_platform = x86_64
+operating_system = GNU/Linux
 
-
-# ============================================================================
-# SSH Specific Options
-# ============================================================================
 [ssh]
-
-# Enable SSH support
-# (default: true)
 enabled = true
-
-
-# Public and private SSH key files. If these don't exist, they are created
-# automatically.
 rsa_public_key = etc/ssh_host_rsa_key.pub
 rsa_private_key = etc/ssh_host_rsa_key
 dsa_public_key = etc/ssh_host_dsa_key.pub
 dsa_private_key = etc/ssh_host_dsa_key
-
-# SSH Version String
-#
-# Use these to disguise your honeypot from a simple SSH version scan
-# Examples:
-# SSH-2.0-OpenSSH_5.1p1 Debian-5
-# SSH-1.99-OpenSSH_4.3
-# SSH-1.99-OpenSSH_4.7
-# SSH-1.99-Sun_SSH_1.1
-# SSH-2.0-OpenSSH_4.2p1 Debian-7ubuntu3.1
-# SSH-2.0-OpenSSH_4.3
-# SSH-2.0-OpenSSH_4.6
-# SSH-2.0-OpenSSH_5.1p1 Debian-5
-# SSH-2.0-OpenSSH_5.1p1 FreeBSD-20080901
-# SSH-2.0-OpenSSH_5.3p1 Debian-3ubuntu5
-# SSH-2.0-OpenSSH_5.3p1 Debian-3ubuntu6
-# SSH-2.0-OpenSSH_5.3p1 Debian-3ubuntu7
-# SSH-2.0-OpenSSH_5.5p1 Debian-6
-# SSH-2.0-OpenSSH_5.5p1 Debian-6+squeeze1
-# SSH-2.0-OpenSSH_5.5p1 Debian-6+squeeze2
-# SSH-2.0-OpenSSH_5.8p2_hpn13v11 FreeBSD-20110503
-# SSH-2.0-OpenSSH_5.9p1 Debian-5ubuntu1
-# SSH-2.0-OpenSSH_6.0p1 Debian-4+deb7u2
-# SSH-2.0-OpenSSH_5.9
-#
-# (default: "SSH-2.0-SSH-2.0-OpenSSH_6.0p1 Debian-4+deb7u2")
 version = SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.2
-
+listen_endpoints = tcp:22:interface=0.0.0.0
-
-# IP addresses to listen for incoming SSH connections.
-#
-# (default: 0.0.0.0) = any IPv4 address
-#listen_addr = 0.0.0.0
-# (use :: for listen to all IPv6 and IPv4 addresses)
-#listen_addr = ::
-
-
-# Port to listen for incoming SSH connections.
-#
-# (default: 2222)
-#listen_port = 22
-
-
-# sftp_enabled enables the sftp subsystem
 sftp_enabled = true
-
-
-# Enable SSH direct-tcpip forwarding
-# (default: true)
 forwarding = true
-
-
-# This enables redirecting forwarding requests to another address
-# Useful for forwarding protocols to other honeypots
-# (default: false)
 forward_redirect = false
+forward_tunnel = false
 
-
-# Configure where to forward the data to.
-# forward_redirect_<portnumber> = <redirect ip>:<redirect port>
-
-# Redirect http/https
-forward_redirect_80 = 127.0.0.1:8000
-forward_redirect_443 = 127.0.0.1:8443
-
-# To record SMTP traffic, install an SMTP honeypoint.
-# (e.g https://github.com/awhitehatter/mailoney), run
-# python mailoney.py -s yahoo.com -t schizo_open_relay -p 12525
-forward_redirect_25 = 127.0.0.1:12525
-forward_redirect_587 = 127.0.0.1:12525
-
-
-
-# ============================================================================
-# Telnet Specific Options
-# ============================================================================
 [telnet]
-
-# Enable Telnet support, disabled by default
 enabled = true
-
+listen_endpoints = tcp:23:interface=0.0.0.0
-# IP addresses to listen for incoming Telnet connections.
-#
-# (default: 0.0.0.0) = any IPv4 address
-#listen_addr = 0.0.0.0
-# (use :: for listen to all IPv6 and IPv4 addresses)
-#listen_addr = ::
-
-
-# Port to listen for incoming Telnet connections.
-#
-# (default: 2223)
-#listen_port = 23
-
-# Source Port to report in logs (useful if you use iptables to forward ports to Cowrie)
 reported_port = 23
 
-
-
-# ============================================================================
-# Database logging Specific Options
-# ============================================================================
-
-# XMPP Logging
-# Log to an xmpp server.
-#
-#[database_xmpp]
-#server = sensors.carnivore.it
-#user = anonymous@sensors.carnivore.it
-#password = anonymous
-#muc = dionaea.sensors.carnivore.it
-#signal_createsession = cowrie-events
-#signal_connectionlost = cowrie-events
-#signal_loginfailed = cowrie-events
-#signal_loginsucceeded = cowrie-events
-#signal_command = cowrie-events
-#signal_clientversion = cowrie-events
-#debug=true
-
-
-
-# ============================================================================
-# Output Plugins
-# These provide an extensible mechanism to send audit log entries to third
-# parties. The audit entries contain information on clients connecting to
-# the honeypot.
-# ============================================================================
-
-
-# JSON based logging module
-#
 [output_jsonlog]
+enabled = true
 logfile = log/cowrie.json
+epoch_timestamp = false
 
-
-# Supports logging to Elasticsearch
-# This is a simple early release
-#
-#[output_elasticsearch]
-#host = localhost
-#port = 9200
-#index = cowrie
-#type = cowrie
-
-
-# Send login attemp information to SANS DShield
-# See https://isc.sans.edu/ssh.html
-# You must signup for an api key.
-# Once registered, find your details at: https://isc.sans.edu/myaccount.html
-#
-#[output_dshield]
-#userid = userid_here
-#auth_key = auth_key_here
-#batch_size = 100
-
-
-# Local Syslog output module
-#
-# This sends log messages to the local syslog daemon.
-# Facility can be:
-# KERN, USER, MAIL, DAEMON, AUTH, LPR, NEWS, UUCP, CRON, SYSLOG and LOCAL0 to LOCAL7.
-#
-# Format can be:
-# text, cef
-#
-#[output_localsyslog]
-#facility = USER
-#format = text
-
-
-# Text output
-# This writes audit log entries to a text file
-#
-# Format can be:
-# text, cef
-#
 [output_textlog]
+enabled = false
 logfile = log/cowrie-textlog.log
 format = text
 
-
-# MySQL logging module
-# Database structure for this module is supplied in doc/sql/mysql.sql
-#
-# MySQL logging requires extra software: sudo apt-get install libmysqlclient-dev
-# MySQL logging requires an extra Python module: pip install mysql-python
-#
-#[output_mysql]
-#host = localhost
-#database = cowrie
-#username = cowrie
-#password = secret
-#port = 3306
-#debug = false
-
-# Rethinkdb output module
-# Rethinkdb output module requires extra Python module: pip install rethinkdb
-
-#[output_rethinkdblog]
-#host = 127.0.0.1
-#port = 28015
-#table = output
-#password =
-#db = cowrie
-
-# SQLite3 logging module
-#
-# Logging to SQLite3 database. To init the database, use the script
-# doc/sql/sqlite3.sql:
-#     sqlite3 <db_file> < doc/sql/sqlite3.sql
-#
-#[output_sqlite]
-#db_file = cowrie.db
-
-# MongoDB logging module
-#
-# MongoDB logging requires an extra Python module: pip install pymongo
-#
-#[output_mongodb]
-#connection_string = mongodb://username:password@host:port/database
-#database = dbname
-
-
-# Splunk SDK output module - Legacy. Requires Splunk API installed
-# This sends logs directly to Splunk using the Python REST SDK
-#
-#[output_splunklegacy]
-#host = localhost
-#port = 8889
-#username = admin
-#password = password
-#index = cowrie
-
-
-# Splunk HTTP Event Collector (HEC) output module
-# Sends JSON directly to Splunk over HTTPS
-# mandatory fields: url, token
-# optional fields: index, source, sourcetype, host
-#
-#[output_splunk]
-#url = https://localhost:8088/services/collector/event
-#token = 6A0EA6C6-8006-4E39-FC44-C35FF6E561A8
-#index = cowrie
-#sourcetype = cowrie
-#source = cowrie
-
-
-# HPFeeds
-#
-#[output_hpfeeds]
-#server = hpfeeds.mysite.org
-#port = 10000
-#identifier = abc123
-#secret = secret
-#debug=false
-
-
-# VirusTotal output module
-# You must signup for an api key.
-#
-#[output_virustotal]
-#api_key = 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
-#
-
-#[output_slack]
-# This will produce a _lot_ of messages - you have been warned....
-#channel = channel_that_events_should_be_posted_in
-#token = slack_token_for_your_bot
-##debug=true
-
-
-# https://csirtg.io
-# You must signup for an api key.
-#
-#[output_csirtg]
-#username=wes
-#feed=scanners
-#description=random scanning activity
-#token=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
-
-
-#[output_socketlog]
-#address = 127.0.0.1:9000
-#timeout = 5

docker/cowrie/dist/userdb.txt

@@ -1 +0,0 @@
-root:0:password

docker/cowrie/docker-compose.yml

@@ -1,6 +1,4 @@
-# T-Pot (Standard)
+version: '2.3'
-# For docker-compose ...
-version: '2.1'
 
 networks:
   cowrie_local:
@@ -9,16 +7,19 @@ services:
 
 # Cowrie service
   cowrie:
+    build: .
     container_name: cowrie
     restart: always
+    tmpfs:
+     - /tmp/cowrie:uid=2000,gid=2000
+     - /tmp/cowrie/data:uid=2000,gid=2000
     networks:
      - cowrie_local
-    cap_add:
-     - NET_BIND_SERVICE
     ports:
-     - "22:2222"
+     - "22:22"
-     - "23:2223"
+     - "23:23"
-    image: "dtagdevsec/cowrie:1710"
+    image: "dtagdevsec/cowrie:1903"
+    read_only: true
     volumes:
      - /data/cowrie/downloads:/home/cowrie/cowrie/dl
      - /data/cowrie/keys:/home/cowrie/cowrie/etc

docker/cyberchef/Dockerfile

@@ -0,0 +1,36 @@
+FROM alpine:3.8
+
+# Get and install dependencies & packages
+RUN apk -U --no-cache add \
+            curl \
+            git \
+            npm \
+            nodejs && \
+    npm install -g grunt-cli && \
+    npm install -g http-server && \
+    npm install npm@latest -g && \
+
+# Install CyberChef 
+    cd /root && \
+    git clone https://github.com/gchq/cyberchef --depth=1 && \
+    chown -R nobody:nobody cyberchef && \
+    cd cyberchef && \
+    npm install && \
+    grunt prod && \
+    mkdir -p /opt/cyberchef && \
+    mv build/prod/* /opt/cyberchef && \
+    cd / && \
+
+# Clean up
+    apk del --purge git \
+                    npm && \
+    rm -rf /root/* && \
+    rm -rf /var/cache/apk/*
+
+# Healthcheck
+HEALTHCHECK --retries=10 CMD curl -s -XGET 'http://127.0.0.1:8000'
+
+# Set user, workdir and start spiderfoot
+USER nobody:nobody
+WORKDIR /opt/cyberchef
+CMD ["http-server", "-p", "8000"]

docker/cyberchef/README.md

@@ -0,0 +1,15 @@
+[![](https://images.microbadger.com/badges/version/dtagdevsec/cyberchef:1903.svg)](https://microbadger.com/images/dtagdevsec/cyberchef:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/cyberchef:1903.svg)](https://microbadger.com/images/dtagdevsec/cyberchef:1903 "Get your own image badge on microbadger.com")
+
+# cyberchef
+
+[cyberchef](https://github.com/gchq/CyberChef) The Cyber Swiss Army Knife - a web app for encryption, encoding, compression and data analysis.
+
+This dockerized version is part of the **[T-Pot community honeypot](http://dtag-dev-sec.github.io/)** of Deutsche Telekom AG.
+
+The `Dockerfile` contains the blueprint for the dockerized version and will be used to setup the docker image.
+
+The `docker-compose.yml` contains the necessary settings to test cyberchef using `docker-compose`. This will ensure to start the docker container with the appropriate permissions and port mappings.
+
+# CyberChef UI
+
+![Cyberchef](doc/dashboard.png)

docker/cyberchef/docker-compose.yml

@@ -0,0 +1,18 @@
+version: '2.3'
+
+networks:
+  cyberchef_local:
+
+services:
+
+# Cyberchef service
+  cyberchef:
+    build: .
+    container_name: cyberchef
+    restart: always
+    networks:
+     - cyberchef_local
+    ports:
+     - "127.0.0.1:64299:8000"
+    image: "dtagdevsec/cyberchef:1903"
+    read_only: true

docker/dionaea/Dockerfile

@@ -1,5 +1,4 @@
 FROM debian:stretch-slim
-MAINTAINER MO
 ENV DEBIAN_FRONTEND noninteractive
 
 # Include dist
@@ -7,15 +6,15 @@ ADD dist/ /root/dist/
 
 # Install dependencies and packages
 RUN apt-get update -y && \
-    apt-get upgrade -y && \
+    apt-get dist-upgrade -y && \
     apt-get install -y --no-install-recommends \
-      autoconf \
-      automake \
 	build-essential \
 	ca-certificates \
 	check \
+	cmake \
 	cython3 \
 	git \
+        libcap2-bin \
 	libcurl4-openssl-dev \
 	libemu-dev \
 	libev-dev \
@@ -31,31 +30,22 @@ RUN apt-get update -y && \
 	python3 \
 	python3-dev \
 	python3-bson \
-      python3-yaml && \
+	python3-yaml \
+	ttf-liberation && \
 
 # Get and install dionaea
-    git clone https://github.com/dinotools/dionaea /root/dionaea/ && \
+    git clone --depth=1 https://github.com/dinotools/dionaea -b 0.8.0 /root/dionaea/ && \
     cd /root/dionaea && \
-#    git checkout 99e9cfc88cfa8f3715813b18ec7006bca2622d76 && \
+    mkdir build && \
-    autoreconf -vi && \
+    cd build && \
-    ./configure \
+    cmake -DCMAKE_INSTALL_PREFIX:PATH=/opt/dionaea .. && \
-      --prefix=/opt/dionaea \
-      --with-python=/usr/bin/python3 \
-      --with-cython-dir=/usr/bin \
-      --enable-ev \
-      --with-ev-include=/usr/include \
-      --with-ev-lib=/usr/lib \
-      --with-emu-lib=/usr/lib/libemu \
-      --with-emu-include=/usr/include \
-      --with-nl-include=/usr/include/libnl3 \
-      --with-nl-lib=/usr/lib \
-      --enable-static && \
     make && \
     make install && \
 
 # Setup user and groups
     addgroup --gid 2000 dionaea && \
     adduser --system --no-create-home --shell /bin/bash --uid 2000 --disabled-password --disabled-login --gid 2000 dionaea && \
+    setcap cap_net_bind_service=+ep /opt/dionaea/bin/dionaea && \
 
 # Supply configs and set permissions
     chown -R dionaea:dionaea /opt/dionaea/var && \
@@ -64,11 +54,10 @@ RUN apt-get update -y && \
 
 # Setup runtime and clean up
     apt-get purge -y \
-      autoconf \
-      automake \
       build-essential \
       ca-certificates \
       check \
+      cmake \
       cython3 \
       git \
       libcurl4-openssl-dev \
@@ -107,4 +96,5 @@ RUN apt-get update -y && \
     rm -rf /root/* /var/lib/apt/lists/* /tmp/* /var/tmp/*
 
 # Start dionaea
+USER dionaea:dionaea
 CMD ["/opt/dionaea/bin/dionaea", "-u", "dionaea", "-g", "dionaea", "-c", "/opt/dionaea/etc/dionaea/dionaea.cfg"]

docker/dionaea/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/dionaea:1710.svg)](https://microbadger.com/images/dtagdevsec/dionaea:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/dionaea:1710.svg)](https://microbadger.com/images/dtagdevsec/dionaea:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/dionaea:1903.svg)](https://microbadger.com/images/dtagdevsec/dionaea:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/dionaea:1903.svg)](https://microbadger.com/images/dtagdevsec/dionaea:1903 "Get your own image badge on microbadger.com")
 
 # dionaea
 
@@ -12,4 +12,4 @@ The `docker-compose.yml` contains the necessary settings to test dionaea using `
 
 # Dionaea Dashboard
 
-![Dionaea Dashboard](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/docker/dionaea/doc/dashboard.png)
+![Dionaea Dashboard](doc/dashboard.png)

docker/dionaea/dist/etc/dionaea.cfg

@@ -8,11 +8,11 @@ listen.mode=getifaddrs
 # listen.interfaces=eth0,tap0
 
 # Country
-ssl.default.c=CN
+ssl.default.c=AU
 # Common Name/domain name
-ssl.default.cn=
+ssl.default.cn=Some-State
 # Organization
-ssl.default.o=
+ssl.default.o=Internet Widgits Pty Ltd
 # Organizational Unit
 ssl.default.ou=
 

docker/dionaea/dist/etc/ihandlers/emu_scripts.yaml

@@ -0,0 +1,9 @@
+- name: emu_scripts
+  config:
+    # Maximum dumber of subdownloads
+    # max_subdownloads: 20
+    enabled_handlers:
+      - powershell
+      - vbscript
+      # Be very careful if you use the raw_url handler
+      #- raw_url

docker/dionaea/dist/etc/services/http.yaml

@@ -2,7 +2,7 @@
   config:
     root: "/opt/dionaea/var/dionaea/roots/www"
     ports:
-      - 80
+      - 81
     ssl_ports:
       - 443
     max_request_size: 32768 # maximum size in kbytes of the request (32MB)

docker/dionaea/dist/etc/services/upnp.yaml

@@ -1,28 +0,0 @@
-- name: upnp
-  config:
-    root: /opt/dionaea/var/dionaea/roots/upnp
-    # maximum size in kbytes of the request (32MB)
-    max_request_size: 32768
-    personality:
-      # default
-      cache:    "CACHE-CONTROL: max-age=120\r\n"
-      st:       "ST: upnp:rootdevice\r\n"
-      usn:      "USN: uuid:Upnp-IPMI-1_0-1234567890001::upnp:rootdevice\r\n"
-      server:   "SERVER: Linux/2.6.17.WB_WPCM450.1.3 UPnP/1.0, Intel SDK for UPnP devices/1.3.1\r\n"
-      location: "LOCATION: http://192.168.0.1:49152/IPMIdevicedesc.xml\r\n"
-      opt:      "OPT: http://schemas.upnp.org/upnp/1/0/\r\n"
-#      # Samsung TV
-#      cache:     "CACHE-CONTROL: max-age=900\r\n"
-#      st:        "ST: uuid:c1fd12b2-d954-4dba-9e92-a697e1558fb4\r\n"
-#      usn:       "USN: uuid:c1fd12b2-d954-4dba-9e92-a697e1558fb4\r\n"
-#      server:    "SERVER: SHP, UPnP/1.0, Samsung UPnP SDK/1.0\r\n"
-#      location:  "LOCATION: http://192.168.0.10:7677/MainTVServer2\r\n"
-#      opt:       "OPT: http://schemas.upnp.org/upnp/1/0/\r\n"
-#
-#      # XBOX 360
-#      cache:     "CACHE-CONTROL: max-age=1800\r\n"
-#      st:        "ST: urn:microsoft.com:service:X_MS_MediaReceiverRegistrar:1\r\n"
-#      usn:       "USN: uuid:531c567a-8c46-4201-bcd4-09afa554d859::urn:microsoft.com:service:X_MS_MediaReceiverRegistrar:1\r\n"
-#      server:    "SERVER: Microsoft-Windows/6.3 UPnP/1.0 UPnP-Device-Host/1.0\r\n"
-#      location:  "LOCATION: http://192.168.0.10:1055/upnphost/udhisapi.dll?content=uuid:531c567a-8c46-4201-bcd4-09afa554d859\r\n"
-#      opt:       "OPT: http://schemas.upnp.org/upnp/1/0/\r\n"

docker/dionaea/docker-compose.yml

@@ -1,40 +1,34 @@
-# T-Pot (Standard)
+version: '2.3'
-# For docker-compose ...
-version: '2.1'
-
-networks:
-  dionaea_local:
 
 services:
 
 # Dionaea service
   dionaea:
+    build: .
     container_name: dionaea
     stdin_open: true
+    tty: true
     restart: always
-    networks:
+    network_mode: "host"
-     - dionaea_local
-    cap_add:
-     - NET_BIND_SERVICE
     ports:
      - "20:20"
      - "21:21"
      - "42:42"
      - "69:69/udp"
-     - "8081:80"
+     - "81:81"
      - "135:135"
      - "443:443"
      - "445:445"
      - "1433:1433"
      - "1723:1723"
      - "1883:1883"
-     - "1900:1900/udp"
      - "3306:3306"
      - "5060:5060"
      - "5060:5060/udp"
      - "5061:5061"
      - "27017:27017"
-    image: "dtagdevsec/dionaea:1710"
+    image: "dtagdevsec/dionaea:1903"
+    read_only: true
     volumes:
      - /data/dionaea/roots/ftp:/opt/dionaea/var/dionaea/roots/ftp
      - /data/dionaea/roots/tftp:/opt/dionaea/var/dionaea/roots/tftp

docker/elasticpot/Dockerfile

@@ -1,30 +1,34 @@
 FROM alpine
-MAINTAINER MS/MO
 
 # Include dist
 ADD dist/ /root/dist/
 
 # Install packages
-RUN apk -U upgrade && \ 
+RUN apk -U --no-cache add \
-    apk add bash python3 git && \
+             git \
-    pip3 install --upgrade pip && \
+             python3 && \
-    pip3 install bottle requests configparser datetime && \
+    pip3 install --no-cache-dir --upgrade pip && \
+    pip3 install --no-cache-dir bottle \
+                                configparser \
+                                datetime \
+                                requests && \
     mkdir -p /opt && \
     cd /opt/ && \
-    git clone https://github.com/schmalle/ElasticpotPY.git && \
+    git clone --depth=1 https://github.com/schmalle/ElasticpotPY.git && \
 
 # Setup user, groups and configs
     addgroup -g 2000 elasticpot && \
-    adduser -S -H -s /bin/bash -u 2000 -D -g 2000 elasticpot && \
+    adduser -S -H -s /bin/ash -u 2000 -D -g 2000 elasticpot && \
     mv /root/dist/elasticpot.cfg /opt/ElasticpotPY/ && \
     mkdir /opt/ElasticpotPY/log && \
 
 # Clean up
-    apk del git && \
+    apk del --purge git && \
     rm -rf /root/* && \
     rm -rf /var/cache/apk/*
 
 # Start elasticpot
-USER elasticpot
+STOPSIGNAL SIGINT
+USER elasticpot:elasticpot
 WORKDIR /opt/ElasticpotPY/
 CMD ["/usr/bin/python3","main.py"]

docker/elasticpot/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/elasticpot:1710.svg)](https://microbadger.com/images/dtagdevsec/elasticpot:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/elasticpot:1710.svg)](https://microbadger.com/images/dtagdevsec/elasticpot:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/elasticpot:1903.svg)](https://microbadger.com/images/dtagdevsec/elasticpot:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/elasticpot:1903.svg)](https://microbadger.com/images/dtagdevsec/elasticpot:1903 "Get your own image badge on microbadger.com")
 
 # elasticpot
 
@@ -12,4 +12,4 @@ The `docker-compose.yml` contains the necessary settings to test elasticpot usin
 
 # ElasticPot Dashboard
 
-![ElasticPot Dashboard](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/docker/elasticpot/doc/dashboard.png)
+![ElasticPot Dashboard](doc/dashboard.png)

docker/elasticpot/docker-compose.yml

@@ -1,4 +1,4 @@
-version: '2.1'
+version: '2.3'
 
 networks:
   elasticpot_local:
@@ -7,12 +7,14 @@ services:
 
 # Elasticpot service
   elasticpot:
+    build: .
     container_name: elasticpot
     restart: always
     networks:
      - elasticpot_local
     ports:
      - "9200:9200"
-    image: "dtagdevsec/elasticpot:1710"
+    image: "dtagdevsec/elasticpot:1903"
+    read_only: true
     volumes:
      - /data/elasticpot/log:/opt/ElasticpotPY/log

docker/elk/README.md

@@ -1,11 +1,11 @@
 # Elasticsearch
-[![](https://images.microbadger.com/badges/version/dtagdevsec/elasticsearch:1710.svg)](https://microbadger.com/images/dtagdevsec/elasticsearch:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/elasticsearch:1710.svg)](https://microbadger.com/images/dtagdevsec/elasticsearch:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/elasticsearch:1903.svg)](https://microbadger.com/images/dtagdevsec/elasticsearch:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/elasticsearch:1903.svg)](https://microbadger.com/images/dtagdevsec/elasticsearch:1903 "Get your own image badge on microbadger.com")
 
 # Logstash
-[![](https://images.microbadger.com/badges/version/dtagdevsec/logstash:1710.svg)](https://microbadger.com/images/dtagdevsec/logstash:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/logstash:1710.svg)](https://microbadger.com/images/dtagdevsec/logstash:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/logstash:1903.svg)](https://microbadger.com/images/dtagdevsec/logstash:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/logstash:1903.svg)](https://microbadger.com/images/dtagdevsec/logstash:1903 "Get your own image badge on microbadger.com")
 
 # Kibana
-[![](https://images.microbadger.com/badges/version/dtagdevsec/kibana:1710.svg)](https://microbadger.com/images/dtagdevsec/kibana:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/kibana:1710.svg)](https://microbadger.com/images/dtagdevsec/kibana:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/kibana:1903.svg)](https://microbadger.com/images/dtagdevsec/kibana:1903 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/kibana:1903.svg)](https://microbadger.com/images/dtagdevsec/kibana:1903 "Get your own image badge on microbadger.com")
 
 # elk stack
 
@@ -17,4 +17,4 @@ The `Dockerfiles` contain the blueprint for the dockerized elk stack and will be
 
 # T-Pot Dashboard
 
-![T-Pot Dashboard](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/docker/elk/doc/dashboard.png)
+![T-Pot Dashboard](doc/dashboard.png)

docker/elk/docker-compose.yml

@@ -0,0 +1,70 @@
+version: '2.3'
+
+services:
+
+# ELK services
+## Elasticsearch service
+  elasticsearch:
+    build: elasticsearch/.
+    container_name: elasticsearch
+    restart: always
+    environment:
+     - bootstrap.memory_lock=true
+     - ES_JAVA_OPTS=-Xms1024m -Xmx1024m
+     - ES_TMPDIR=/tmp
+    cap_add:
+     - IPC_LOCK
+    ulimits:
+      memlock:
+        soft: -1
+        hard: -1
+      nofile:
+        soft: 65536
+        hard: 65536
+    mem_limit: 4g
+    ports:
+     - "127.0.0.1:64298:9200"
+    image: "dtagdevsec/elasticsearch:1903"
+    volumes:
+     - /data:/data
+
+## Kibana service
+  kibana:
+    build: kibana/.
+    container_name: kibana
+    restart: always
+    stop_signal: SIGKILL
+    depends_on:
+      elasticsearch:
+        condition: service_healthy
+    ports:
+     - "127.0.0.1:64296:5601"
+    image: "dtagdevsec/kibana:1903"
+
+## Logstash service
+  logstash:
+    build: logstash/.
+    container_name: logstash
+    restart: always
+    depends_on:
+      elasticsearch:
+        condition: service_healthy
+    env_file:
+     - /opt/tpot/etc/compose/elk_environment
+    image: "dtagdevsec/logstash:1903"
+    volumes:
+     - /data:/data
+     - /root/tpotce/docker/elk/logstash/dist/logstash.conf:/etc/logstash/conf.d/logstash.conf
+
+## Elasticsearch-head service
+  head:
+    build: head/.
+    container_name: head
+    restart: always
+    depends_on:
+      elasticsearch:
+        condition: service_healthy
+    ports:
+     - "127.0.0.1:64302:9100"
+    image: "dtagdevsec/head:1903"
+    read_only: true

docker/elk/elasticsearch/Dockerfile

@@ -1,18 +1,22 @@
 FROM alpine
-MAINTAINER MO
 
 # Include dist
 ADD dist/ /root/dist/
 
 # Setup env and apt
-RUN apk -U upgrade && \
+RUN sed -i 's/dl-cdn/dl-2/g' /etc/apk/repositories && \
-    apk add bash curl openjdk8-jre procps wget && \
+    apk -U --no-cache add \
+             aria2 \
+             bash \
+             curl \
+             nss \
+             openjdk8-jre && \
 
 # Get and install packages
     cd /root/dist/ && \
     mkdir -p /usr/share/elasticsearch/ && \
-    wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.3.tar.gz && \
+    aria2c -s 16 -x 16 https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.6.2.tar.gz && \
-    tar xvfz elasticsearch-5.6.3.tar.gz --strip-components=1 -C /usr/share/elasticsearch/ && \
+    tar xvfz elasticsearch-6.6.2.tar.gz --strip-components=1 -C /usr/share/elasticsearch/ && \
 
 # Add and move files
     cd /root/dist/ && \
@@ -21,16 +25,19 @@ RUN apk -U upgrade && \
 
 # Setup user, groups and configs
     addgroup -g 2000 elasticsearch && \
-    adduser -S -H -s /bin/bash -u 2000 -D -g 2000 elasticsearch && \
+    adduser -S -H -s /bin/ash -u 2000 -D -g 2000 elasticsearch && \
     chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/ && \
+    rm -rf /usr/share/elasticsearch/modules/x-pack-ml && \
 
 # Clean up
-    apk del wget && \
+    apk del --purge aria2 && \
-    rm -rf /root/*
+    rm -rf /root/* && \
+    rm -rf /tmp/* && \
+    rm -rf /var/cache/apk/*
 
 # Healthcheck
 HEALTHCHECK --retries=10 CMD curl -s -XGET 'http://127.0.0.1:9200/_cat/health'
 
 # Start ELK
-USER elasticsearch
+USER elasticsearch:elasticsearch
 CMD ["/usr/share/elasticsearch/bin/elasticsearch"]