diff --git a/README.md b/README.md index 74a67319..22261c87 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ ![T-Pot](doc/tpotsocial.png) -T-Pot 19.03 runs on Debian (Stable), is based heavily on +T-Pot 20.06 runs on Debian (Stable), is based heavily on [docker](https://www.docker.com/), [docker-compose](https://docs.docker.com/compose/) @@ -12,7 +12,7 @@ and includes dockerized versions of the following honeypots * [conpot](http://conpot.org/), * [cowrie](https://github.com/cowrie/cowrie), * [dionaea](https://github.com/DinoTools/dionaea), -* [elasticpot](https://github.com/schmalle/ElasticpotPY), +* [elasticpot](https://gitlab.com/bontchev/elasticpot), * [glutton](https://github.com/mushorg/glutton), * [heralding](https://github.com/johnnykv/heralding), * [honeypy](https://github.com/foospidy/HoneyPy), @@ -88,7 +88,7 @@ In T-Pot we combine the dockerized honeypots ... * [conpot](http://conpot.org/), * [cowrie](http://www.micheloosterhof.com/cowrie/), * [dionaea](https://github.com/DinoTools/dionaea), -* [elasticpot](https://github.com/schmalle/ElasticpotPY), +* [elasticpot](https://gitlab.com/bontchev/elasticpot), * [glutton](https://github.com/mushorg/glutton), * [heralding](https://github.com/johnnykv/heralding), * [honeypy](https://github.com/foospidy/HoneyPy), @@ -179,7 +179,7 @@ Depending on your installation type, whether you install on [real hardware](#har # Installation The installation of T-Pot is straight forward and heavily depends on a working, transparent and non-proxied up and running internet connection. Otherwise the installation **will fail!** -Firstly, decide if you want to download our prebuilt installation ISO image from [GitHub](https://github.com/dtag-dev-sec/tpotce/releases), [create it yourself](#createiso) ***or*** [post-install on an existing Debian 9.7 (Stretch)](#postinstall). +Firstly, decide if you want to download our prebuilt installation ISO image from [GitHub](https://github.com/dtag-dev-sec/tpotce/releases), [create it yourself](#createiso) ***or*** [post-install on an existing Debian 10 (Buster)](#postinstall). Secondly, decide where you want to let the system run: [real hardware](#hardware) or in a [virtual machine](#vm)? @@ -193,7 +193,7 @@ You can download the prebuilt installation image from [GitHub](https://github.co For transparency reasons and to give you the ability to customize your install, we provide you the [ISO Creator](https://github.com/dtag-dev-sec/tpotce) that enables you to create your own ISO installation image. **Requirements to create the ISO image:** -- Debian 9.7 or newer as host system (others *may* work, but *remain* untested) +- Debian 10 as host system (others *may* work, but *remain* untested) - 4GB of free memory - 32GB of free storage - A working internet connection @@ -240,7 +240,7 @@ Whereas most CD burning tools allow you to burn from ISO images, the procedure t ## Post-Install User -In some cases it is necessary to install Debian 9.7 (Stretch) on your own: +In some cases it is necessary to install Debian 10 (Buster) on your own: - Cloud provider does not offer mounting ISO images. - Hardware setup needs special drivers and / or kernels. - Within your company you have to setup special policies, software etc. @@ -474,7 +474,7 @@ We hope you understand that we cannot provide support on an individual basis. We # Licenses The software that T-Pot is built on uses the following licenses.
GPLv2: [conpot](https://github.com/mushorg/conpot/blob/master/LICENSE.txt), [dionaea](https://github.com/DinoTools/dionaea/blob/master/LICENSE), [honeypy](https://github.com/foospidy/HoneyPy/blob/master/LICENSE), [honeytrap](https://github.com/armedpot/honeytrap/blob/master/LICENSE), [suricata](http://suricata-ids.org/about/open-source/) -
GPLv3: [adbhoney](https://github.com/huuck/ADBHoney), [elasticpot](https://github.com/schmalle/ElasticpotPY), [ewsposter](https://github.com/dtag-dev-sec/ews/), [fatt](https://github.com/0x4D31/fatt/blob/master/LICENSE), [rdpy](https://github.com/citronneur/rdpy/blob/master/LICENSE), [heralding](https://github.com/johnnykv/heralding/blob/master/LICENSE.txt), [snare](https://github.com/mushorg/snare/blob/master/LICENSE), [tanner](https://github.com/mushorg/snare/blob/master/LICENSE) +
GPLv3: [adbhoney](https://github.com/huuck/ADBHoney), [elasticpot](https://gitlab.com/bontchev/elasticpot/-/blob/master/LICENSE), [ewsposter](https://github.com/dtag-dev-sec/ews/), [fatt](https://github.com/0x4D31/fatt/blob/master/LICENSE), [rdpy](https://github.com/citronneur/rdpy/blob/master/LICENSE), [heralding](https://github.com/johnnykv/heralding/blob/master/LICENSE.txt), [snare](https://github.com/mushorg/snare/blob/master/LICENSE), [tanner](https://github.com/mushorg/snare/blob/master/LICENSE)
Apache 2 License: [cyberchef](https://github.com/gchq/CyberChef/blob/master/LICENSE), [elasticsearch](https://github.com/elasticsearch/elasticsearch/blob/master/LICENSE.txt), [logstash](https://github.com/elasticsearch/logstash/blob/master/LICENSE), [kibana](https://github.com/elasticsearch/kibana/blob/master/LICENSE.md), [docker](https://github.com/docker/docker/blob/master/LICENSE), [elasticsearch-head](https://github.com/mobz/elasticsearch-head/blob/master/LICENCE)
MIT license: [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot/blob/master/LICENSE), [glutton](https://github.com/mushorg/glutton/blob/master/LICENSE)
Other: [citrixhoneypot](https://github.com/MalwareTech/CitrixHoneypot#licencing-agreement-malwaretech-public-licence), [cowrie](https://github.com/micheloosterhof/cowrie/blob/master/LICENSE.md), [mailoney](https://github.com/awhitehatter/mailoney), [Debian licensing](https://www.debian.org/legal/licenses/) @@ -495,7 +495,7 @@ Without open source and the fruitful development community (we are proud to be a * [debian](http://www.debian.org/) * [dionaea](https://github.com/DinoTools/dionaea/graphs/contributors) * [docker](https://github.com/docker/docker/graphs/contributors) -* [elasticpot](https://github.com/schmalle/ElasticpotPY/graphs/contributors) +* [elasticpot](https://gitlab.com/bontchev/elasticpot/-/project_members) * [elasticsearch](https://github.com/elastic/elasticsearch/graphs/contributors) * [elasticsearch-head](https://github.com/mobz/elasticsearch-head/graphs/contributors) * [ewsposter](https://github.com/armedpot/ewsposter/graphs/contributors) diff --git a/bin/hptest.sh b/bin/hptest.sh index 48a96b9c..3500fd56 100755 --- a/bin/hptest.sh +++ b/bin/hptest.sh @@ -83,6 +83,7 @@ fuCHECKFORARGS echo "Starting scans ..." echo "$myMEDPOTPACKET" | nc "$myHOST" 2575 & curl -XGET "http://$myHOST:9200/logstash-*/_search" & +curl -XPOST -H "Content-Type: application/json" -d '{"name":"test","email":"test@test.com"}' "http://$myHOST:9200/test" & echo "I20100" | timeout --foreground 3 nc "$myHOST" 10001 & fuSCAN "180" "7,8,102,135,161,1025,1080,5000,9200" "$myHOST" "-sC -sS -sU -sV" fuSCAN "180" "2048,4096,5432" "$myHOST" "-sC -sS -sU -sV --version-light" diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml index 1a0bb2c4..fa5703a7 100644 --- a/docker/docker-compose.yml +++ b/docker/docker-compose.yml @@ -37,6 +37,11 @@ services: build: dionaea/. image: "dtagdevsec/dionaea:2006" +# ElasticPot service + elasticpot: + build: elasticpot/. + image: "dtagdevsec/elasticpot:2006" + # Glutton service glutton: build: glutton/. diff --git a/docker/elasticpot.old/Dockerfile b/docker/elasticpot.old/Dockerfile new file mode 100644 index 00000000..42b2579c --- /dev/null +++ b/docker/elasticpot.old/Dockerfile @@ -0,0 +1,34 @@ +FROM alpine:latest +# +# Include dist +ADD dist/ /root/dist/ +# +# Install packages +RUN apk -U --no-cache add \ + git \ + py3-pip \ + python3 && \ + pip3 install --no-cache-dir bottle \ + configparser \ + datetime \ + requests && \ + mkdir -p /opt && \ + cd /opt/ && \ + git clone --depth=1 https://github.com/schmalle/ElasticpotPY.git && \ +# +# Setup user, groups and configs + addgroup -g 2000 elasticpot && \ + adduser -S -H -s /bin/ash -u 2000 -D -g 2000 elasticpot && \ + mv /root/dist/elasticpot.cfg /opt/ElasticpotPY/ && \ + mkdir /opt/ElasticpotPY/log && \ +# +# Clean up + apk del --purge git && \ + rm -rf /root/* && \ + rm -rf /var/cache/apk/* +# +# Start elasticpot +STOPSIGNAL SIGINT +USER elasticpot:elasticpot +WORKDIR /opt/ElasticpotPY/ +CMD ["/usr/bin/python3","main.py"] diff --git a/docker/elasticpot/README.md b/docker/elasticpot.old/README.md similarity index 100% rename from docker/elasticpot/README.md rename to docker/elasticpot.old/README.md diff --git a/docker/elasticpot/dist/elasticpot.cfg b/docker/elasticpot.old/dist/elasticpot.cfg similarity index 100% rename from docker/elasticpot/dist/elasticpot.cfg rename to docker/elasticpot.old/dist/elasticpot.cfg diff --git a/docker/elasticpot/doc/dashboard.png b/docker/elasticpot.old/doc/dashboard.png similarity index 100% rename from docker/elasticpot/doc/dashboard.png rename to docker/elasticpot.old/doc/dashboard.png diff --git a/docker/elasticpot.old/docker-compose.yml b/docker/elasticpot.old/docker-compose.yml new file mode 100644 index 00000000..a8fd3547 --- /dev/null +++ b/docker/elasticpot.old/docker-compose.yml @@ -0,0 +1,20 @@ +version: '2.3' + +networks: + elasticpot_local: + +services: + +# Elasticpot service + elasticpot: + build: . + container_name: elasticpot + restart: always + networks: + - elasticpot_local + ports: + - "9200:9200" + image: "dtagdevsec/elasticpot:2006" + read_only: true + volumes: + - /data/elasticpot/log:/opt/ElasticpotPY/log diff --git a/docker/elasticpot/Dockerfile b/docker/elasticpot/Dockerfile index 42b2579c..590c9c2f 100644 --- a/docker/elasticpot/Dockerfile +++ b/docker/elasticpot/Dockerfile @@ -4,31 +4,41 @@ FROM alpine:latest ADD dist/ /root/dist/ # # Install packages -RUN apk -U --no-cache add \ +RUN sed -i 's/dl-cdn/dl-2/g' /etc/apk/repositories && \ + apk -U add \ + build-base \ + ca-certificates \ git \ + libffi-dev \ + openssl \ + openssl-dev \ + py3-mysqlclient \ + py3-requests \ py3-pip \ - python3 && \ - pip3 install --no-cache-dir bottle \ - configparser \ - datetime \ - requests && \ + python3 \ + python3-dev && \ mkdir -p /opt && \ cd /opt/ && \ - git clone --depth=1 https://github.com/schmalle/ElasticpotPY.git && \ + git clone --depth=1 https://gitlab.com/bontchev/elasticpot.git/ && \ + cd elasticpot && \ + pip3 install -r requirements.txt && \ # # Setup user, groups and configs addgroup -g 2000 elasticpot && \ adduser -S -H -s /bin/ash -u 2000 -D -g 2000 elasticpot && \ - mv /root/dist/elasticpot.cfg /opt/ElasticpotPY/ && \ - mkdir /opt/ElasticpotPY/log && \ + mv /root/dist/honeypot.cfg /opt/elasticpot/etc/ && \ # # Clean up - apk del --purge git && \ + apk del --purge build-base \ + git \ + libffi-dev \ + openssl-dev \ + python3-dev && \ rm -rf /root/* && \ rm -rf /var/cache/apk/* # # Start elasticpot STOPSIGNAL SIGINT USER elasticpot:elasticpot -WORKDIR /opt/ElasticpotPY/ -CMD ["/usr/bin/python3","main.py"] +WORKDIR /opt/elasticpot/ +CMD ["/usr/bin/python3","elasticpot.py"] diff --git a/docker/elasticpot/dist/honeypot.cfg b/docker/elasticpot/dist/honeypot.cfg new file mode 100644 index 00000000..865d4369 --- /dev/null +++ b/docker/elasticpot/dist/honeypot.cfg @@ -0,0 +1,301 @@ +# ============================================================================ +# General Honeypot Options +# ============================================================================ +[honeypot] + +# Sensor name is used to identify this honeypot instance. Used by the database +# logging modules such as JSON. +# +# If not specified, the logging modules will instead use the IP address of the +# server as the sensor name. +# +# (default: the name of the local machine) +sensor_name = t-pot + +# The version of Elasticsearch reported by the honeypot. +# +# (default: 1.4.1) +#spoofed_version = 1.4.1 + +# The Elasticsearch instance name reported by the honeypot. +# +# (default = Green Goblin) +instance_name = USNYES01 + +# The name of the simulated Elasticsearch cluster +# +# (default = elasticsearch +#cluster_name = elasticsearch + +# The name of the simulated host running Elasticsearch +# +# (default = elk) +host_name = usnyes01 + +# The build number of the simulated Elasticsearch instance +# Use something realistic or simply don't touch this value +# +# (default = 89d3241) +#build = 89d3241 + +# The number of processors on the simulated host +# +# (default = 12) +#total_processors = 12 + +# The total number of CPU cores on the simulated host +# Use a multiple of total_processors +# +# (default = 24) +#total_cores = 24 + +# The total number of sockets on the simulated host +# Use a multiple of total_cores +# +# (default = 48) +#total_sockets = 48 + +# The MAC address of the networking card of the simulated host +# +# (default = 08:01:c7:3F:15:DD) +#mac_address = 08:01:c7:3F:15:DD + +# Directory where to save log files in. +# Log files are .YYYY-MM-DD in that directory +# +# (default: log) +log_path = log + +# Log file name +# +# (default: stdout) +#log_filename = + +# Directory containing the response files +# +# (default: responses) +#responses_dir = responses + +# ============================================================================ +# Network Specific Options +# ============================================================================ + +# Port to listen for incoming connections. +# +# (default: 9200) +#listen_port = 9200 + +# Site to query for one's public IP address +# +# (default: https://ident.me) +#public_ip_url = https://ident.me + +# Enable to log the public IP of the honeypot (useful if listening on 127.0.0.1) +# IP address is obtained by querying public_ip_url +# +# (default: false) +#report_public_ip = false + + +# ============================================================================ +# Output Plugins +# These provide an extensible mechanism to send audit log entries to third +# parties. The audit entries contain information on clients connecting to +# the honeypot. +# +# Output entries need to start with 'output_' and have the 'enabled' entry. +# ============================================================================ + +# JSON based logging module +# +[output_jsonlog] +enabled = true +logfile = log/elasticpot.json +epoch_timestamp = false + +# MySQL logging module +# Database structure for this module is supplied in docs/sql/mysql.sql +# +# MySQL logging requires extra software: sudo apt-get install libmysqlclient-dev +# MySQL logging requires an extra Python module: pip install mysql-python +# +#[output_mysql] +#enabled = false +#host = localhost +#database = elasticpot +#username = elasticpot +#password = secret +#port = 3306 +#debug = false +# Whether to store geolocation data in the database +#geoip = true +# Location of the databases used for geolocation +#geoip_citydb = data/GeoLite2-City.mmdb +#geoip_asndb = data/GeoLite2-ASN.mmdb + +# Text output +# This writes audit log entries to a text file +# +#[output_textlog] +#enabled = false +#logfile = log/elasticpot.txt + +# HPFeeds +# +# Note the lack of "s" at the end: +[output_hpfeed] +enabled = false +#server = hpfeeds.mysite.org +#tlscert = /path/to/tls/cert/file +#port = 10000 +#identifier = abc123 +#secret = secret +#channel = elasticpot + +# MongoDB logging module +# +#[output_mongodb] +#enabled = false +#host = 127.0.0.1 +#port = 27017 +#username = elasticpot +#password = secret +#database = elasticpot +# Note: .format(username, password, host, port, database) is done +# on the following string; make sure that there are 5 placeholders ({}) in it +#connection_string = mongodb://{}:{}@{}:{}/{} +# Whether to store geolocation data in the database +#geoip = true +# Location of the databases used for geolocation +#geoip_citydb = data/GeoLite2-City.mmdb +#geoip_asndb = data/GeoLite2-ASN.mmdb + +# RedisDB logging module +# +#[output_redisdb] +#enabled = false +#host = 127.0.0.1 +#port = 6379 +# DB of the redis server. Defaults to 0 +#db = 0 +# Password of the redis server. Defaults to None +#password = secret +# Name of the list to push to or the channel to publish to. Required +#keyname = elasticpot +# Method to use when sending data to redis. +# Can be one of [lpush, rpush, publish]. Defaults to lpush +#send_method = lpush + +# Rethinkdb output module +# +#[output_rethinkdblog] +#enabled = false +#host = 127.0.0.1 +#port = 28015 +#table = events +#db = elasticpot +#password = + +# InfluxDB logging module +# +#[output_influx] +#enabled = false +#host = 127.0.0.1 +#port = 8086 +#database_name = elasticpot +#retention_policy_duration = 12w + +# InfluxDB 2.0 logging module +# +#[output_influx2] +#enabled = false +#host = hostname +#token = token +#org = organization +#bucket = elasticpot + +# CouchDB logging module +# +#[output_couch] +#enabled = false +#host = localhost +#port = 5984 +#username = elasticpot +#password = secret +#database = elasticpot +#geoip = true +# Location of the databases used for geolocation +#geoip_citydb = data/GeoLite2-City.mmdb +#geoip_asndb = data/GeoLite2-ASN.mmdb + +# SQLite3 logging module +# +# Logging to SQLite3 database. To init the database, use the script +# docs/sql/sqlite3.sql: +# sqlite3 < docs/sql/sqlite3.sql +# +#[output_sqlite] +#enabled = false +#debug = false +#db_file = data/elasticpot.db +# Whether to store geolocation data in the database +#geoip = true +# Location of the databases used for geolocation +#geoip_citydb = data/GeoLite2-City.mmdb +#geoip_asndb = data/GeoLite2-ASN.mmdb + +# Elasticsearch logging module +# +#[output_elastic] +#enabled = false +#host = localhost +#port = 9200 +#index = elasticpot +# +# type has been deprecated since ES 6.0.0 +# use _doc which is the default type. See +# https://stackoverflow.com/a/53688626 for +# more information +# +#type = _doc +# +# set pipeline = geoip to map src_ip to +# geo location data. You can use a custom +# pipeline but you must ensure it exists +# in elasticsearch. +# +#pipeline = geoip +# +# Authentication. When x-pack.security is enabled +# in ES, default users have been created and requests +# must be authenticated. +# +# Credentials +# +#username = elasticpot +#password = secret +# +# TLS encryption. Communications between the client (elasticpot) +# and the ES server should naturally be protected by encryption +# if requests are authenticated (to prevent from man-in-the-middle +# attacks). The following options are then paramount +# if username and password are provided. +# +# use ssl/tls +#ssl = true +# verify SSL certificates +#verify_certs = true +# Path to trusted CA certs on disk +#ca_certs = /path/to/cert/file/elastic_ca.crt + + +# TODO: + +# Kafka logging module +# +#[output_kafka] +#enabled = false +#host = 127.0.0.1 +#port = 9092 +#topic = elasticpot + diff --git a/docker/elasticpot/docker-compose.yml b/docker/elasticpot/docker-compose.yml index a8fd3547..16ce22cf 100644 --- a/docker/elasticpot/docker-compose.yml +++ b/docker/elasticpot/docker-compose.yml @@ -17,4 +17,4 @@ services: image: "dtagdevsec/elasticpot:2006" read_only: true volumes: - - /data/elasticpot/log:/opt/ElasticpotPY/log + - /data/elasticpot/log:/opt/elasticpot/log diff --git a/docker/elk/logstash/dist/logstash.conf b/docker/elk/logstash/dist/logstash.conf index 0beb0c10..c7e251f5 100644 --- a/docker/elk/logstash/dist/logstash.conf +++ b/docker/elk/logstash/dist/logstash.conf @@ -64,9 +64,9 @@ input { type => "Dionaea" } -# Elasticpot +# ElasticPot file { - path => ["/data/elasticpot/log/elasticpot.log"] + path => ["/data/elasticpot/log/elasticpot.json"] codec => json type => "ElasticPot" } @@ -296,6 +296,17 @@ filter { date { match => [ "timestamp", "ISO8601" ] } + mutate { + rename => { + "content_type" => "http.http_content_type" + "dst_port" => "dest_port" + "dst_ip" => "dest_ip" + "message" => "event_type" + "request" => "request_method" + "user_agent" => "http_user_agent" + "url" => "http.url" + } + } } # Glutton diff --git a/etc/compose/nextgen.yml b/etc/compose/nextgen.yml index d1174a99..c702d92f 100644 --- a/etc/compose/nextgen.yml +++ b/etc/compose/nextgen.yml @@ -11,6 +11,7 @@ networks: conpot_local_kamstrup_382: cowrie_local: cyberchef_local: + elasticpot_local: heralding_local: honeypy_local: mailoney_local: @@ -210,6 +211,19 @@ services: - /data/dionaea/log:/opt/dionaea/var/log - /data/dionaea/rtp:/opt/dionaea/var/dionaea/rtp +# ElasticPot service + elasticpot: + container_name: elasticpot + restart: always + networks: + - elasticpot_local + ports: + - "9200:9200" + image: "dtagdevsec/elasticpot:2006" + read_only: true + volumes: + - /data/elasticpot/log:/opt/elasticpot/log + # Glutton service glutton: container_name: glutton @@ -268,7 +282,7 @@ services: - "2323:2323" - "2324:2324" - "4096:4096" - - "9200:9200" + # - "9200:9200" image: "dtagdevsec/honeypy:2006" read_only: true volumes: diff --git a/etc/compose/sensor.yml b/etc/compose/sensor.yml index dcd61db6..83e103b2 100644 --- a/etc/compose/sensor.yml +++ b/etc/compose/sensor.yml @@ -195,7 +195,7 @@ services: - /data/dionaea/log:/opt/dionaea/var/log - /data/dionaea/rtp:/opt/dionaea/var/dionaea/rtp -# Elasticpot service +# ElasticPot service elasticpot: container_name: elasticpot restart: always @@ -206,7 +206,7 @@ services: image: "dtagdevsec/elasticpot:2006" read_only: true volumes: - - /data/elasticpot/log:/opt/ElasticpotPY/log + - /data/elasticpot/log:/opt/elasticpot/log # Heralding service heralding: diff --git a/etc/compose/standard.yml b/etc/compose/standard.yml index ffebd3d4..2a869b88 100644 --- a/etc/compose/standard.yml +++ b/etc/compose/standard.yml @@ -196,7 +196,7 @@ services: - /data/dionaea/log:/opt/dionaea/var/log - /data/dionaea/rtp:/opt/dionaea/var/dionaea/rtp -# Elasticpot service +# ElasticPot service elasticpot: container_name: elasticpot restart: always @@ -207,7 +207,7 @@ services: image: "dtagdevsec/elasticpot:2006" read_only: true volumes: - - /data/elasticpot/log:/opt/ElasticpotPY/log + - /data/elasticpot/log:/opt/elasticpot/log # Heralding service heralding: diff --git a/etc/logrotate/logrotate.conf b/etc/logrotate/logrotate.conf index 9f42e53c..6059cc91 100644 --- a/etc/logrotate/logrotate.conf +++ b/etc/logrotate/logrotate.conf @@ -11,6 +11,7 @@ /data/dionaea/log/dionaea.sqlite /data/dionaea/dionaea-errors.log /data/elasticpot/log/elasticpot.log +/data/elasticpot/log/elasticpot.json /data/elk/log/*.log /data/fatt/log/fatt.log /data/glutton/log/*.log