Update folder naming

cloud/ansible/.gitignore

@@ -0,0 +1,2 @@
+# Ansible
+*.retry

cloud/ansible/README.md

@@ -0,0 +1,228 @@
+# Ansible T-Pot Deployment on Open Telekom Cloud :cloud:
+
+Here you can find a ready-to-use solution for your automated T-Pot deployment using [Ansible](https://www.ansible.com/).  
+It consists of multiple Ansible Playbooks, which can be reused across all Cloud Providers (like AWS, Azure, Digital Ocean).  
+This example showcases the deployment on our own Public Cloud Offering [Open Telekom Cloud](https://open-telekom-cloud.com/en).
+
+# Table of contents
+- [Installation of Ansible Master](#installation)
+  - [Packages](#packages)
+  - [Agent Forwarding](#agent-forwarding)
+- [Preparations in Open Telekom Cloud Console](#preparation)
+  - [Create new project](#project)
+  - [Create API user](#api-user)
+  - [Import Key Pair](#key-pair)
+  - [Create VPC, Subnet and Security Group](#vpc-subnet-securitygroup)
+- [Clone Git Repository](#clone-git)
+- [Settings and recommended values](#settings)
+  - [Configure `.otc_env.sh`](#otc-env)
+  - [Configure `.ecs_settings.sh`](#ecs-settings)
+  - [Configure `tpot.conf.dist`](#tpot-conf)
+  - [Optional: Custom `ews.cfg` and HPFEEDS](#ews-hpfeeds)
+- [Deploying a T-Pot](#deploy)
+- [Further documentation](#documentation)
+
+<a name="installation"></a>
+# Installation of Ansible Master
+You can either run the deploy script locally on your Linux or MacOS machine or you can use an ECS (Elastic Cloud Server) on Open Telekom Cloud, which I did.  
+I used Ubuntu 18.04 for my Ansible Master Server, but other OSes are fine too.  
+Ansible works over the SSH Port, so you don't have to add any special rules to your Security Group.
+
+<a name="packages"></a>
+## Packages
+At first we need to add the repository and install Ansible:  
+`sudo apt-add-repository --yes --update ppa:ansible/ansible`  
+`sudo apt install ansible`
+
+Also we need **pwegen** (for creating T-Pot names) and **jq** (a JSON processor):  
+`sudo apt install pwgen jq`
+
+<a name="agent-forwarding"></a>
+## Agent Forwarding
+Agent forwarding must be enabled in order to let Ansible do its work.  
+- On Linux or MacOS:  
+  - Create or edit `~/.ssh/config`
+  - If you execute the script remotely on your Ansible Master Server:
+    ```
+    Host ANSIBLE_MASTER_IP
+    ForwardAgent yes
+    ```
+  - If you execute the script locally, enable it for all Hosts, as this includes newly generated T-Pots:
+    ```
+    Host *
+    ForwardAgent yes
+    ```
+- On Windows using Putty:  
+![Putty Agent Forwarding](doc/putty_agent_forwarding.png)
+
+<a name="preparation"></a>
+# Preparations in Open Telekom Cloud Console
+(You can skip this if you have already set up an API account, VPC and ...)  
+(Just make sure you know the naming for everything, as you will need it to configure the script.)
+
+Before we can start deploying, we have to prepare the Open Telekom Cloud Tennant.  
+For that, go to the [Web Console](https://auth.otc.t-systems.com/authui/login) and log in with an admin user.
+
+<a name="project"></a>
+## Create new project
+I strongly advise you, to create a separate project for the T-Pots in your tennant.  
+In my case I named it `tpot`.
+
+![Create new project](doc/otc_1_project.gif)
+
+<a name="api-user"></a>
+## Create API user
+The next step is to create a new user account, which is restricted to the project.  
+This ensures that the API access is limited to that project.
+
+![Create API user](doc/otc_2_user.gif)
+
+<a name="key-pair"></a>
+## Import Key Pair
+:warning: Now log in with the newly created user account and select your project.
+
+![Login as API user](doc/otc_3_login.gif)
+
+
+Import your SSH public key.
+
+![Import SSH Public Key](doc/otc_4_import_key.gif)
+
+<a name="vpc-subnet-securitygroup"></a>
+## Create VPC, Subnet and Security Group
+- VPC (Virtual Private Cloud) and Subnet:
+
+![Create VPC and Subnet](doc/otc_5_vpc_subnet.gif)
+
+- Security Group:  
+The configured Security Group should allow all incoming TCP / UDP traffic.  
+If you want to secure the management interfaces, you can limit the incoming "allow all" traffic to the port range of 1-64000 and allow access to ports > 64000 only from your trusted IPs.
+
+![Create Security Group](doc/otc_6_sec_group.gif)
+
+<a name="clone-git"></a>
+# Clone Git Repository
+Clone the `tpotce` repository to your Ansible Master:  
+`git clone https://github.com/dtag-dev-sec/tpotce.git`  
+All Ansible and automatic deployment related files are located in the [`cloud/open-telekom-cloud`](../../cloud/open-telekom-cloud) folder.
+
+<a name="settings"></a>
+# Settings and recommended values
+You can configure all aspects of your ECS and T-Pot before using the script.  
+The settings are located in the following files:
+
+<a name="otc-env"></a>
+## Configure `.otc_env.sh`
+Enter your Open Telekom Cloud API user credentials here (username, password, tennant-ID, project name):  
+```
+export OS_USERNAME=your_api_user
+export OS_PASSWORD=your_password
+export OS_USER_DOMAIN_NAME=OTC-EU-DE-000000000010000XXXXX
+export OS_PROJECT_NAME=eu-de_your_project
+export OS_AUTH_URL=https://iam.eu-de.otc.t-systems.com/v3
+```
+
+<a name="ecs-settings"></a>
+## Configure `.ecs_settings.sh`
+Here you can customize your Elastic Cloud Server (ECS):
+  - Password for the user `linux` (**you should definitely change that**)  
+    You may have to adjust the `remote_user` in the Ansible Playbooks under [ansible](ansible) if you are using a normal/default Debian base image
+  - (Optional) For using a custom `ews.cfg` set to `true`; See here: [Optional: Custom `ews.cfg`](#ews-cfg)
+  - (Optional) Change the instance type (flavor) of the ECS.  
+    `s2.medium.8` corresponds to 1 vCPU and 8GB of RAM and is the minimum required flavor.  
+    A full list of flavors can be found [here](https://docs.otc.t-systems.com/en-us/usermanual/ecs/en-us_topic_0035470096.html).
+  - Change the OS (Don't touch; for T-Pot we need Debian 9)
+  - Specify the VPC, Subnet, Security Group and Key Pair you created before
+  - (Optional) Change the disk size
+  - You can choose from multiple Availibility Zones (AZ). For reference see [here](https://docs.otc.t-systems.com/en-us/endpoint/index.html).
+
+```
+# Set password for user linux
+linuxpass=LiNuXuSeRPaSs#
+
+# Custom EWS config
+custom_ews=false
+
+# Set ECS related stuff
+instance=s2.medium.8
+imagename=Standard_Debian_9_latest
+subnet=your-subnet
+vpcname=your-vpc
+secgroup=your-sg
+keyname=your-KeyPair
+disksize=128
+az=eu-de-03
+```
+
+<a name="tpot-conf"></a>
+## Configure `tpot.conf.dist`
+The file is located in [`iso/installer/tpot.conf.dist`](../../iso/installer/tpot.conf.dist).  
+Here you can choose:
+  - between the various T-Pot editions
+  - a username for the web interface
+  - a password for the web interface (**you should definitely change that**)
+
+```
+# tpot configuration file
+# myCONF_TPOT_FLAVOR=[STANDARD, SENSOR, INDUSTRIAL, COLLECTOR, NEXTGEN]
+myCONF_TPOT_FLAVOR='STANDARD'
+myCONF_WEB_USER='webuser'
+myCONF_WEB_PW='w3b$ecret'
+```
+
+<a name="ews-hpfeeds"></a>
+## Optional: Custom `ews.cfg` and HPFEEDS
+To enable these features, set `custom_ews=true` in `.ecs_settings.sh`; See here:  [Configure `.ecs_settings.sh`](#ecs-settings)  
+
+### ews.cfg
+You can use a custom config file for `ewsposter`.  
+e.g. when you have your own credentials for delivering data to our [Sicherheitstacho](https://sicherheitstacho.eu/start/main).  
+You can find the `ews.cfg` template file here: [`ansible/roles/custom_ews/templates/ews.cfg`](ansible/roles/custom_ews/templates/ews.cfg) and adapt it for your needs.
+
+For setting custom credentials, these settings would be relevant for you (the rest of the file can stay as is):  
+```
+[MAIN]
+...
+contact = your_email_address
+...
+
+[EWS]
+...
+username = your_username
+token = your_token
+...
+```
+
+### HPFEEDS
+You can also specify HPFEEDS in [`ansible/roles/custom_ews/templates/hpfeeds.cfg`](ansible/roles/custom_ews/templates/hpfeeds.cfg).  
+That file constains the defaults (turned off) and you can adapt it for your needs, e.g. for SISSDEN:
+```
+myENABLE=true
+myHOST=hpfeeds.sissden.eu
+myPORT=10000
+myCHANNEL=t-pot.events
+myCERT=/opt/ewsposter/sissden.pem
+myIDENT=your_user
+mySECRET=your_secret
+myFORMAT=json
+```
+
+
+<a name="deploy"></a>
+# Deploying a T-Pot :honey_pot::honeybee:
+Now, after configuring everything, we can finally start deploying T-Pots:  
+`./deploy_ansible_otc_t-pot.sh`  
+(Yes, it is as easy as that :smile:)
+
+The script will first create an Open Telekom Cloud ECS via the API.  
+After that, the Ansible Playbooks are executed on the newly created Host to install the T-Pot and configure everything.
+
+You can see the progress of every step in the console output.  
+If something should go wrong, you will be provided with an according error message, that you can hopefully act upon and retry.
+
+<a name="documentation"></a>
+# Further documentation
+- [Ansible Documentation](https://docs.ansible.com/ansible/latest/)
+- [Open Telekom Cloud Help Center](https://docs.otc.t-systems.com/)
+- [Open Telekom Cloud API Overview](https://docs.otc.t-systems.com/en-us/api/wp/en-us_topic_0052070394.html)
+- [otc-tools](https://github.com/OpenTelekomCloud/otc-tools) on GitHub

cloud/ansible/openstack/ansible.cfg

@@ -0,0 +1,5 @@
+[defaults]
+host_key_checking = false
+
+[ssh_connection]
+scp_if_ssh = true

cloud/ansible/openstack/deploy_tpot.yaml

@@ -0,0 +1,25 @@
+- name: Check host prerequisites
+  hosts: localhost
+  become: yes
+  become_user: root
+  become_method: sudo
+  roles:
+    - check
+
+- name: Deploy instance
+  hosts: localhost
+  roles:
+    - deploy
+
+- name: Install T-Pot on new instance
+  hosts: TPOT
+  remote_user: linux
+  become: yes
+  become_user: root
+  become_method: sudo
+  gather_facts: no
+  roles:
+    - install
+#    - custom_ews
+#    - custom_hpfeeds
+    - reboot

cloud/ansible/openstack/roles/check/tasks/main.yaml

@@ -0,0 +1,25 @@
+- name: Install pwgen
+  apt:
+    name: pwgen
+
+- name: Install setuptools
+  apt:
+    name: python-setuptools
+
+- name: Install pip
+  apt:
+    name: python-pip
+
+- name: Install openstacksdk
+  pip:
+    name: openstacksdk
+
+- name: Set fact for agent forwarding
+  set_fact:
+    agent_forwarding: "{{ lookup('env','SSH_AUTH_SOCK') }}"
+
+- name: Check if agent forwarding is enabled
+  fail:
+    msg: Please enable agent forwarding to allow Ansible to connect to the remote host!
+  ignore_errors: yes
+  when: agent_forwarding == ""

cloud/ansible/openstack/roles/custom_ews/tasks/main.yaml

@@ -0,0 +1,13 @@
+- name: Copy ews configuration file
+  template:
+    src: ../templates/ews.cfg
+    dest: /data/ews/conf
+    owner: root
+    group: root
+    mode: 0644
+
+- name: Patching tpot.yml with custom ews configuration file
+  lineinfile:
+    path: /opt/tpot/etc/tpot.yml
+    insertafter: '/opt/ewsposter/ews.ip'
+    line: '     - /data/ews/conf/ews.cfg:/opt/ewsposter/ews.cfg'

cloud/ansible/openstack/roles/custom_ews/templates/ews.cfg

@@ -0,0 +1,137 @@
+[MAIN]
+homedir = /opt/ewsposter/
+spooldir = /opt/ewsposter/spool/
+logdir = /opt/ewsposter/log/
+del_malware_after_send = false
+send_malware = true
+sendlimit = 500
+contact = your_email_address
+proxy =
+ip =
+
+[EWS]
+ews = true
+username = your_username
+token = your_token
+rhost_first = https://community.sicherheitstacho.eu/ews-0.1/alert/postSimpleMessage
+rhost_second = https://community.sicherheitstacho.eu/ews-0.1/alert/postSimpleMessage
+ignorecert = false
+
+[HPFEED]
+hpfeed = %(EWS_HPFEEDS_ENABLE)s
+host = %(EWS_HPFEEDS_HOST)s
+port = %(EWS_HPFEEDS_PORT)s
+channels = %(EWS_HPFEEDS_CHANNELS)s
+ident = %(EWS_HPFEEDS_IDENT)s
+secret= %(EWS_HPFEEDS_SECRET)s
+# path/to/certificate for tls broker - or "false" for non-tls broker
+tlscert = %(EWS_HPFEEDS_TLSCERT)s
+# hpfeeds submission format: "ews" (xml) or "json"
+hpfformat = %(EWS_HPFEEDS_FORMAT)s
+
+[EWSJSON]
+json = false
+jsondir = /data/ews/json/
+
+[GLASTOPFV3]
+glastopfv3 = true
+nodeid = glastopfv3-{{ ansible_hostname }}
+sqlitedb = /data/glastopf/db/glastopf.db
+malwaredir = /data/glastopf/data/files/
+
+[GLASTOPFV2]
+glastopfv2 = false
+nodeid =
+mysqlhost =
+mysqldb =
+mysqluser =
+mysqlpw =
+malwaredir =
+
+[KIPPO]
+kippo = false
+nodeid =
+mysqlhost =
+mysqldb =
+mysqluser =
+mysqlpw =
+malwaredir =
+
+[COWRIE]
+cowrie = true
+nodeid = cowrie-{{ ansible_hostname }}
+logfile = /data/cowrie/log/cowrie.json
+
+[DIONAEA]
+dionaea = true
+nodeid = dionaea-{{ ansible_hostname }}
+malwaredir = /data/dionaea/binaries/
+sqlitedb = /data/dionaea/log/dionaea.sqlite
+
+[HONEYTRAP]
+honeytrap = true
+nodeid = honeytrap-{{ ansible_hostname }}
+newversion = true
+payloaddir = /data/honeytrap/attacks/
+attackerfile = /data/honeytrap/log/attacker.log
+
+[RDPDETECT]
+rdpdetect = false
+nodeid =
+iptableslog =
+targetip =
+
+[EMOBILITY]
+eMobility = false
+nodeid = emobility-{{ ansible_hostname }}
+logfile = /data/emobility/log/centralsystemEWS.log
+
+[CONPOT]
+conpot = true
+nodeid = conpot-{{ ansible_hostname }}
+logfile = /data/conpot/log/conpot*.json
+
+[ELASTICPOT]
+elasticpot = true
+nodeid = elasticpot-{{ ansible_hostname }}
+logfile = /data/elasticpot/log/elasticpot.log
+
+[SURICATA]
+suricata = true
+nodeid = suricata-{{ ansible_hostname }}
+logfile = /data/suricata/log/eve.json
+
+[MAILONEY]
+mailoney = true
+nodeid = mailoney-{{ ansible_hostname }}
+logfile = /data/mailoney/log/commands.log
+
+[RDPY]
+rdpy = true
+nodeid = rdpy-{{ ansible_hostname }}
+logfile = /data/rdpy/log/rdpy.log
+
+[VNCLOWPOT]
+vnclowpot = true
+nodeid = vnclowpot-{{ ansible_hostname }}
+logfile = /data/vnclowpot/log/vnclowpot.log
+
+[HERALDING]
+heralding = true
+nodeid = heralding-{{ ansible_hostname }}
+logfile = /data/heralding/log/auth.csv
+
+[CISCOASA]
+ciscoasa = true
+nodeid = ciscoasa-{{ ansible_hostname }}
+logfile = /data/ciscoasa/log/ciscoasa.log
+
+[TANNER]
+tanner = true
+nodeid = tanner-{{ ansible_hostname }}
+logfile = /data/tanner/log/tanner_report.json
+
+[GLUTTON]
+glutton = true
+nodeid = glutton-{{ ansible_hostname }}
+logfile = /data/glutton/log/glutton.log

cloud/ansible/openstack/roles/custom_hpfeeds/tasks/main.yaml

@@ -0,0 +1,10 @@
+- name: Copy hpfeeds configuration file
+  template:
+    src: ../templates/hpfeeds.cfg
+    dest: /data/ews/conf
+    owner: root
+    group: root
+    mode: 0644
+
+- name: Applying hpfeeds settings
+  command: /opt/tpot/bin/hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg

cloud/ansible/openstack/roles/custom_hpfeeds/templates/hpfeeds.cfg

@@ -0,0 +1,8 @@
+myENABLE=false
+myHOST=host
+myPORT=port
+myCHANNEL=channels
+myCERT=false
+myIDENT=user
+mySECRET=secret
+myFORMAT=json

cloud/ansible/openstack/roles/deploy/tasks/main.yaml

@@ -0,0 +1,34 @@
+- name: Create T-Pot name
+  shell: echo t-pot-otc-$(pwgen -ns 6 -1)
+  register: tpot_name
+
+- name: Import OpenStack authentication variables
+  include_vars:
+    file: roles/deploy/vars/os_auth.yaml
+
+- name: Launch an instance
+  os_server:
+    auth:
+      auth_url: "{{ auth_url }}"
+      username: "{{ username }}"
+      password: "{{ password }}"
+      project_name: "{{ project_name }}"
+      os_user_domain_name: "{{ os_user_domain_name }}"
+    name: "{{ tpot_name.stdout }}"
+    region_name: "{{ region_name }}"
+    availability_zone: "{{ availability_zone }}"
+    image: "{{ image }}"
+    boot_from_volume: yes
+    volume_size: "{{ volume_size }}"
+    key_name: "{{ key_name }}"
+    timeout: 200
+    flavor: "{{ flavor }}"
+    security_groups: "{{ security_groups }}"
+    network: "{{ network }}"
+  register: tpot
+
+- name: Add instance to inventory
+  add_host:
+    hostname: "{{ tpot_name.stdout }}"
+    ansible_host: "{{ tpot.server.public_v4 }}"
+    groups: TPOT

cloud/ansible/openstack/roles/deploy/vars/main.yaml

@@ -0,0 +1,8 @@
+region_name: eu-de
+availability_zone: eu-de-03
+image: Standard_Debian_9_latest
+volume_size: 128
+key_name: your-KeyPair
+flavor: s2.medium.8
+security_groups: your-sg
+network: your-network-id

cloud/ansible/openstack/roles/deploy/vars/os_auth.yaml

@@ -0,0 +1,5 @@
+auth_url: https://iam.eu-de.otc.t-systems.com/v3
+username: your_api_user
+password: your_password
+project_name: eu-de_your_project
+os_user_domain_name: OTC-EU-DE-000000000010000XXXXX

cloud/ansible/openstack/roles/install/tasks/main.yaml

@@ -0,0 +1,50 @@
+- name: Waiting for SSH connection
+  wait_for_connection:
+    delay: 30
+    timeout: 300
+
+- name: Gathering Facts
+  setup:
+
+- name: Cloning t-pot install directory
+  git:
+    repo: "https://github.com/dtag-dev-sec/tpotce.git"
+    dest: /root/tpot
+
+- name: Prepare to set user password 
+  set_fact:
+    user_password: "{{ linux_pass }}"
+    user_salt: "s0mew1ck3dTpoT"
+
+- name: Changing password for user linux to {{ user_password }}
+  user:
+   name: "linux"
+   password: "{{ user_password | password_hash('sha512', user_salt) }}"
+   state: present
+   shell: /bin/bash
+   update_password: always
+
+- name: Copy t-pot configuration file
+  template:
+    src: ../../../../../../iso/installer/tpot.conf.dist
+    dest: /root/tpot.conf
+    owner: root
+    group: root
+    mode: 0644
+
+- name: Install t-pot on ECS -  be patient, this might take 15 to 30 minutes depending on the connection speed. No further output is given. 
+  command: /root/tpot/iso/installer/install.sh --type=auto --conf=/root/tpot.conf
+
+- name: Delete t-pot configuration file
+  file:
+    path: /root/tpot.conf
+    state: absent
+
+- name: Change unattended-upgrades to take default action
+  blockinfile:
+    dest: /etc/apt/apt.conf.d/50unattended-upgrades
+    block: |
+      Dpkg::Options {
+        "--force-confdef";
+        "--force-confold";
+      }

cloud/ansible/openstack/roles/install/vars/main.yaml

@@ -0,0 +1 @@
+linux_pass: LiNuXuSeRPaSs#

cloud/ansible/openstack/roles/reboot/tasks/main.yaml

@@ -0,0 +1,3 @@
+- name: Finally rebooting t-pot in one minute - make sure your next login is on port 64295 or via https:// on port 64297
+  shell: /sbin/shutdown -r -t 1
+  become: true