Merge branch 'master' of https://github.com/telekom-security/tpotce

README.md

@@ -24,6 +24,7 @@ and includes dockerized versions of the following honeypots
 * [honeysap](https://github.com/SecureAuthCorp/HoneySAP),
 * [honeytrap](https://github.com/armedpot/honeytrap/),
 * [ipphoney](https://gitlab.com/bontchev/ipphoney),
+* [log4pot](https://github.com/thomaspatzke/Log4Pot),
 * [mailoney](https://github.com/awhitehatter/mailoney),
 * [medpot](https://github.com/schmalle/medpot),
 * [rdpy](https://github.com/citronneur/rdpy),
@@ -110,6 +111,7 @@ In T-Pot we combine the dockerized honeypots ...
 * [honeysap](https://github.com/SecureAuthCorp/HoneySAP),
 * [honeytrap](https://github.com/armedpot/honeytrap/),
 * [ipphoney](https://gitlab.com/bontchev/ipphoney),
+* [log4pot](https://github.com/thomaspatzke/Log4Pot),
 * [mailoney](https://github.com/awhitehatter/mailoney),
 * [medpot](https://github.com/schmalle/medpot),
 * [redishoneypot](https://github.com/cypwnpwnsocute/RedisHoneyPot),
@@ -499,7 +501,7 @@ We hope you understand that we cannot provide support on an individual basis. We
 # Licenses
 The software that T-Pot is built on uses the following licenses.
 <br>GPLv2: [conpot](https://github.com/mushorg/conpot/blob/master/LICENSE.txt), [dionaea](https://github.com/DinoTools/dionaea/blob/master/LICENSE), [honeysap](https://github.com/SecureAuthCorp/HoneySAP/blob/master/COPYING), [honeypy](https://github.com/foospidy/HoneyPy/blob/master/LICENSE), [honeytrap](https://github.com/armedpot/honeytrap/blob/master/LICENSE), [suricata](http://suricata-ids.org/about/open-source/)
-<br>GPLv3: [adbhoney](https://github.com/huuck/ADBHoney), [elasticpot](https://gitlab.com/bontchev/elasticpot/-/blob/master/LICENSE), [ewsposter](https://github.com/telekom-security/ews/), [fatt](https://github.com/0x4D31/fatt/blob/master/LICENSE), [rdpy](https://github.com/citronneur/rdpy/blob/master/LICENSE), [heralding](https://github.com/johnnykv/heralding/blob/master/LICENSE.txt), [ipphoney](https://gitlab.com/bontchev/ipphoney/-/blob/master/LICENSE), [redishoneypot](https://github.com/cypwnpwnsocute/RedisHoneyPot/blob/main/LICENSE), [snare](https://github.com/mushorg/snare/blob/master/LICENSE), [tanner](https://github.com/mushorg/snare/blob/master/LICENSE)
+<br>GPLv3: [adbhoney](https://github.com/huuck/ADBHoney), [elasticpot](https://gitlab.com/bontchev/elasticpot/-/blob/master/LICENSE), [ewsposter](https://github.com/telekom-security/ews/), [log4pot](https://github.com/thomaspatzke/Log4Pot/blob/master/LICENSE), [fatt](https://github.com/0x4D31/fatt/blob/master/LICENSE), [rdpy](https://github.com/citronneur/rdpy/blob/master/LICENSE), [heralding](https://github.com/johnnykv/heralding/blob/master/LICENSE.txt), [ipphoney](https://gitlab.com/bontchev/ipphoney/-/blob/master/LICENSE), [redishoneypot](https://github.com/cypwnpwnsocute/RedisHoneyPot/blob/main/LICENSE), [snare](https://github.com/mushorg/snare/blob/master/LICENSE), [tanner](https://github.com/mushorg/snare/blob/master/LICENSE)
 <br>Apache 2 License: [cyberchef](https://github.com/gchq/CyberChef/blob/master/LICENSE), [dicompot](https://github.com/nsmfoo/dicompot/blob/master/LICENSE), [elasticsearch](https://github.com/elasticsearch/elasticsearch/blob/master/LICENSE.txt), [logstash](https://github.com/elasticsearch/logstash/blob/master/LICENSE), [kibana](https://github.com/elasticsearch/kibana/blob/master/LICENSE.md), [docker](https://github.com/docker/docker/blob/master/LICENSE), [elasticsearch-head](https://github.com/mobz/elasticsearch-head/blob/master/LICENCE)
 <br>MIT license: [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot/blob/master/LICENSE), [ddospot](https://github.com/aelth/ddospot/blob/master/LICENSE), [glutton](https://github.com/mushorg/glutton/blob/master/LICENSE), [hellpot](https://github.com/yunginnanet/HellPot/blob/master/LICENSE)
 <br> Unlicense: [endlessh](https://github.com/skeeto/endlessh/blob/master/UNLICENSE)
@@ -541,6 +543,7 @@ Without open source and the fruitful development community (we are proud to be a
 * [ipphoney](https://gitlab.com/bontchev/ipphoney/-/project_members)
 * [kibana](https://github.com/elastic/kibana/graphs/contributors)
 * [logstash](https://github.com/elastic/logstash/graphs/contributors)
+* [log4pot](https://github.com/thomaspatzke/Log4Pot/graphs/contributors)
 * [mailoney](https://github.com/awhitehatter/mailoney)
 * [medpot](https://github.com/schmalle/medpot/graphs/contributors)
 * [p0f](http://lcamtuf.coredump.cx/p0f3/)

bin/clean.sh

@@ -237,6 +237,14 @@ fuIPPHONEY () {
   chown tpot:tpot /data/ipphoney -R
 }
 
+# Let's create a function to clean up and prepare log4pot data
+fuLOG4POT () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/log4pot/*; fi
+  mkdir -p /data/log4pot/log
+  chmod 770 /data/log4pot -R
+  chown tpot:tpot /data/log4pot -R
+}
+
 # Let's create a function to clean up and prepare mailoney data
 fuMAILONEY () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/mailoney/*; fi
@@ -351,6 +359,7 @@ if [ "$myPERSISTENCE" = "on" ];
     fuHONEYPY
     fuHONEYTRAP
     fuIPPHONEY
+    fuLOG4POT
     fuMAILONEY
     fuMEDPOT
     fuNGINX

docker/elk/elasticsearch/Dockerfile

@@ -1,7 +1,7 @@
 FROM alpine:3.14
 #
 # VARS
-ENV ES_VER=7.15.1 \
+ENV ES_VER=7.16.2 \
     ES_JAVA_HOME=/usr/lib/jvm/java-16-openjdk
 
 # Include dist
@@ -14,7 +14,7 @@ RUN apk -U --no-cache add \
              bash \
              curl \
              nss && \
-    apk add --no-cache -X http://dl-cdn.alpinelinux.org/alpine/edge/testing openjdk16-jre && \
+    apk add --no-cache -X http://dl-cdn.alpinelinux.org/alpine/edge/community openjdk16-jre && \
 #
 # Get and install packages
     cd /root/dist/ && \

docker/elk/kibana/Dockerfile

@@ -1,7 +1,7 @@
-FROM node:14.17.6-alpine3.14
+FROM node:16.13.0-alpine3.14
 #
 # VARS
-ENV KB_VER=7.15.1
+ENV KB_VER=7.16.2
 # 
 # Include dist
 ADD dist/ /root/dist/

docker/elk/logstash/dist/logstash.conf

@@ -147,6 +147,13 @@ input {
     type => "Ipphoney"
   }
 
+# Log4pot
+  file {
+    path => ["/data/log4pot/log/log4pot.log"]
+    codec => json
+    type => "Log4pot"
+  }
+
 # Mailoney
   file {
     path => ["/data/mailoney/log/commands.log"]
@@ -564,6 +571,20 @@ filter {
     }
   }
 
+# Log4pot
+  if [type] == "Log4pot" {
+    date {
+      match => [ "timestamp", "ISO8601" ]
+    }
+    mutate {
+      rename => {
+        "server_port" => "dest_port"
+        "port" => "src_port"
+        "client" => "src_ip"
+      }
+    }
+  }
+
 # Mailoney
   if [type] == "Mailoney" {
     date {
@@ -649,12 +670,12 @@ if "_jsonparsefailure" in [tags] { drop {} }
     geoip {
       cache_size => 10000
       source => "src_ip"
-      database => "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip-7.2.3-java/vendor/GeoLite2-City.mmdb"
+      database => "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip-7.2.8-java/vendor/GeoLite2-City.mmdb"
     }
     geoip {
       cache_size => 10000
       source => "src_ip"
-      database => "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip-7.2.3-java/vendor/GeoLite2-ASN.mmdb"
+      database => "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip-7.2.8-java/vendor/GeoLite2-ASN.mmdb"
     }
     translate {
       refresh_interval => 86400

docker/log4pot/Dockerfile

@@ -0,0 +1,58 @@
+FROM ubuntu:20.04
+ENV DEBIAN_FRONTEND noninteractive
+#
+# Install packages
+RUN apt-get update && \
+    apt-get update -y && \
+    apt-get dist-upgrade -y && \
+    apt-get install -y \
+             build-essential \
+	     cargo \
+	     cleo \
+             git \
+	     libcap2 \
+	     libcap2-bin \
+	     libcurl4 \
+	     libcurl4-nss-dev \
+	     libffi7 \
+	     libffi-dev \
+	     libssl-dev \
+	     python3-pip \
+             python3 \
+             python3-dev \
+             rust-all && \
+     pip3 install --upgrade pip && \
+     pip3 install poetry pycurl && \
+#	     
+# Install log4pot from GitHub and setup
+    mkdir -p /opt /var/log/log4pot && \
+    cd /opt/ && \
+    git clone https://github.com/thomaspatzke/Log4Pot && \
+    cd Log4Pot && \
+#    git checkout 4269bf4a91457328fb64c3e7941cb2f520e5e911 && \
+    git checkout 4e9bac32605e4d2dd4bbc6df56365988b4815c4a && \
+    sed -i 's#"type": logtype,#"reason": logtype,#g' log4pot.py && \
+    poetry install && \
+    setcap cap_net_bind_service=+ep /usr/bin/python3.8 && \
+#
+# Setup user, groups and configs
+    addgroup --gid 2000 log4pot && \
+    adduser --system --no-create-home --shell /bin/bash -uid 2000 --disabled-password --disabled-login -gid 2000 log4pot && \
+    chown log4pot:log4pot -R /opt/Log4Pot && \
+#
+# Clean up
+     apt-get purge -y build-essential \
+                    cargo \
+                    git \
+		    libffi-dev \
+		    libssl-dev \
+		    python3-dev \
+		    rust-all && \
+    apt-get autoremove -y --purge && \
+    apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
+#
+# Start log4pot
+STOPSIGNAL SIGINT
+USER log4pot:log4pot
+WORKDIR /opt/Log4Pot/
+CMD ["/usr/bin/python3","log4pot.py","--port","8080","--log","/var/log/log4pot/log/log4pot.log","--download-dir","/var/log/log4pot/payloads/","--download-class","--download-payloads"]

docker/log4pot/docker-compose.yml

@@ -0,0 +1,27 @@
+version: '2.3'
+
+networks:
+  log4pot_local:
+
+services:
+
+# Log4pot service
+  log4pot:
+    build: .
+    container_name: log4pot
+    restart: always
+    tmpfs:
+     - /tmp:uid=2000,gid=2000
+    networks:
+     - log4pot_local
+    ports:
+     - "80:8080"
+     - "443:8080"
+     - "8080:8080"
+     - "9200:8080"
+     - "25565:8080"
+    image: "dtagdevsec/log4pot:2006"
+    read_only: true
+    volumes:
+     - /data/log4pot/log:/var/log/log4pot/log
+     - /data/log4pot/payloads:/var/log/log4pot/payloads

etc/compose/log4j.yml

@@ -0,0 +1,241 @@
+# T-Pot (Log4j)
+# Do not erase ports sections, these are used by /opt/tpot/bin/rules.sh to setup iptables ACCEPT rules for NFQ (honeytrap / glutton)
+version: '2.3'
+
+networks:
+  cyberchef_local:
+  log4pot_local:
+  ewsposter_local:
+  spiderfoot_local:
+
+services:
+
+##################
+#### Honeypots
+##################
+
+# Log4pot service
+  log4pot:
+    container_name: log4pot
+    restart: always
+    tmpfs:
+     - /tmp:uid=2000,gid=2000
+    networks:
+     - log4pot_local
+    ports:
+     - "80:8080"
+     - "443:8080"
+     - "8080:8080"
+     - "9200:8080"
+     - "25565:8080"
+    image: "dtagdevsec/log4pot:2006"
+    read_only: true
+    volumes:
+     - /data/log4pot/log:/var/log/log4pot/log
+     - /data/log4pot/payloads:/var/log/log4pot/payloads
+
+# Honeytrap service
+  honeytrap:
+    container_name: honeytrap
+    restart: always
+    tmpfs:
+     - /tmp/honeytrap:uid=2000,gid=2000
+    network_mode: "host"
+    cap_add:
+     - NET_ADMIN
+    image: "dtagdevsec/honeytrap:2006"
+    read_only: true
+    volumes:
+     - /data/honeytrap/attacks:/opt/honeytrap/var/attacks
+     - /data/honeytrap/downloads:/opt/honeytrap/var/downloads
+     - /data/honeytrap/log:/opt/honeytrap/var/log
+
+
+##################
+#### NSM
+##################
+
+# Fatt service
+  fatt:
+    container_name: fatt
+    restart: always
+    network_mode: "host"
+    cap_add:
+     - NET_ADMIN
+     - SYS_NICE
+     - NET_RAW
+    image: "dtagdevsec/fatt:2006"
+    volumes:
+     - /data/fatt/log:/opt/fatt/log
+
+# P0f service
+  p0f:
+    container_name: p0f
+    restart: always
+    network_mode: "host"
+    image: "dtagdevsec/p0f:2006"
+    read_only: true
+    volumes:
+     - /data/p0f/log:/var/log/p0f
+
+# Suricata service
+  suricata:
+    container_name: suricata
+    restart: always
+    environment:
+    # For ET Pro ruleset replace "OPEN" with your OINKCODE
+     - OINKCODE=OPEN
+    network_mode: "host"
+    cap_add:
+     - NET_ADMIN
+     - SYS_NICE
+     - NET_RAW
+    image: "dtagdevsec/suricata:2006"
+    volumes:
+     - /data/suricata/log:/var/log/suricata
+
+
+##################
+#### Tools
+##################
+
+# Cyberchef service
+  cyberchef:
+    container_name: cyberchef
+    restart: always
+    networks:
+     - cyberchef_local
+    ports:
+     - "127.0.0.1:64299:8000"
+    image: "dtagdevsec/cyberchef:2006"
+    read_only: true
+
+#### ELK
+## Elasticsearch service
+  elasticsearch:
+    container_name: elasticsearch
+    restart: always
+    environment:
+     - bootstrap.memory_lock=true
+#     - ES_JAVA_OPTS=-Xms2048m -Xmx2048m
+     - ES_TMPDIR=/tmp
+    cap_add:
+     - IPC_LOCK
+    ulimits:
+      memlock:
+        soft: -1
+        hard: -1
+      nofile:
+        soft: 65536
+        hard: 65536
+#    mem_limit: 4g
+    ports:
+     - "127.0.0.1:64298:9200"
+    image: "dtagdevsec/elasticsearch:2006"
+    volumes:
+     - /data:/data
+
+## Kibana service
+  kibana:
+    container_name: kibana
+    restart: always
+    depends_on:
+      elasticsearch:
+        condition: service_healthy
+    ports:
+     - "127.0.0.1:64296:5601"
+    image: "dtagdevsec/kibana:2006"
+
+## Logstash service
+  logstash:
+    container_name: logstash
+    restart: always
+#    environment:
+#     - LS_JAVA_OPTS=-Xms2048m -Xmx2048m
+    depends_on:
+      elasticsearch:
+        condition: service_healthy
+    env_file:
+     - /opt/tpot/etc/compose/elk_environment
+    image: "dtagdevsec/logstash:2006"
+    volumes:
+     - /data:/data
+
+## Elasticsearch-head service
+  head:
+    container_name: head
+    restart: always
+    depends_on:
+      elasticsearch:
+        condition: service_healthy
+    ports:
+     - "127.0.0.1:64302:9100"
+    image: "dtagdevsec/head:2006"
+    read_only: true
+
+# Ewsposter service
+  ewsposter:
+    container_name: ewsposter
+    restart: always
+    networks:
+     - ewsposter_local
+    environment:
+     - EWS_HPFEEDS_ENABLE=false
+     - EWS_HPFEEDS_HOST=host
+     - EWS_HPFEEDS_PORT=port
+     - EWS_HPFEEDS_CHANNELS=channels
+     - EWS_HPFEEDS_IDENT=user
+     - EWS_HPFEEDS_SECRET=secret
+     - EWS_HPFEEDS_TLSCERT=false
+     - EWS_HPFEEDS_FORMAT=json
+    env_file:
+     - /opt/tpot/etc/compose/elk_environment
+    image: "dtagdevsec/ewsposter:2006"
+    volumes:
+     - /data:/data
+     - /data/ews/conf/ews.ip:/opt/ewsposter/ews.ip
+
+# Nginx service
+  nginx:
+    container_name: nginx
+    restart: always
+    environment:
+    ### If set to YES all changes within Heimdall will remain for the next start
+    ### Make sure to uncomment the corresponding volume statements below, or the setting will prevent a successful start of T-Pot.
+     - HEIMDALL_PERSIST=NO
+    tmpfs:
+     - /var/tmp/nginx/client_body
+     - /var/tmp/nginx/proxy
+     - /var/tmp/nginx/fastcgi
+     - /var/tmp/nginx/uwsgi
+     - /var/tmp/nginx/scgi 
+     - /run
+     - /var/log/php7/
+     - /var/lib/nginx/tmp:uid=100,gid=82 
+     - /var/lib/nginx/html/storage/logs:uid=100,gid=82
+     - /var/lib/nginx/html/storage/framework/views:uid=100,gid=82
+    network_mode: "host"
+    ports:
+     - "64297:64297"
+     - "127.0.0.1:64304:64304"
+    image: "dtagdevsec/nginx:2006"
+    read_only: true
+    volumes:
+     - /data/nginx/cert/:/etc/nginx/cert/:ro
+     - /data/nginx/conf/nginxpasswd:/etc/nginx/nginxpasswd:ro
+     - /data/nginx/log/:/var/log/nginx/
+    ### Enable the following volumes if you set HEIMDALL_PERSIST=YES
+    # - /data/nginx/heimdall/database:/var/lib/nginx/html/database
+    # - /data/nginx/heimdall/storage:/var/lib/nginx/html/storage
+
+# Spiderfoot service
+  spiderfoot:
+    container_name: spiderfoot
+    restart: always
+    networks:
+     - spiderfoot_local
+    ports:
+     - "127.0.0.1:64303:8080"
+    image: "dtagdevsec/spiderfoot:2006"
+    volumes:
+     - /data/spiderfoot/spiderfoot.db:/home/spiderfoot/spiderfoot.db

etc/logrotate/logrotate.conf

@@ -29,6 +29,7 @@
 /data/honeytrap/log/*.log
 /data/honeytrap/log/*.json
 /data/ipphoney/log/*.json
+/data/log4pot/log/*.log
 /data/mailoney/log/*.log
 /data/medpot/log/*.log
 /data/nginx/log/*.log

iso/installer/install.sh

@@ -159,10 +159,10 @@ ListenStream=
 ListenStream=64294
 "
 mySSHSETTINGS="
+Port 64295
 Match Group tpotlogs
         PermitOpen 127.0.0.1:64305
         ForceCommand /usr/bin/false
-Port 64295
 "
 myRANDOM_HOUR=$(shuf -i 2-22 -n 1)
 myRANDOM_MINUTE=$(shuf -i 0-59 -n 1)
@@ -515,8 +515,9 @@ fi
 if [ "$myTPOT_DEPLOYMENT_TYPE" == "iso" ] || [ "$myTPOT_DEPLOYMENT_TYPE" == "user" ];
   then
     myCONF_TPOT_FLAVOR=$(dialog --keep-window --no-cancel --backtitle "$myBACKTITLE" --title "[ Choose Your T-Pot Edition ]" --menu \
-    "\nRequired: 8GB RAM, 128GB SSD\nRecommended: 8GB RAM, 256GB SSD" 15 70 6 \
+    "\nRequired: 8GB RAM, 128GB SSD\nRecommended: 8GB RAM, 256GB SSD" 15 70 7 \
     "STANDARD" "Honeypots, ELK, NSM & Tools" \
+    "LOG4J" "Log4Pot, ELK, NSM & Tools" \
     "SENSOR" "Just Honeypots, EWS Poster & NSM" \
     "INDUSTRIAL" "Conpot, RDPY, Vnclowpot, ELK, NSM & Tools" \
     "COLLECTOR" "Heralding, ELK, NSM & Tools" \
@@ -731,6 +732,10 @@ case $myCONF_TPOT_FLAVOR in
     fuBANNER "STANDARD"
     ln -s /opt/tpot/etc/compose/standard.yml $myTPOTCOMPOSE
   ;;
+  LOG4J)
+    fuBANNER "LOG4J"
+    ln -s /opt/tpot/etc/compose/log4j.yml $myTPOTCOMPOSE
+  ;;
   SENSOR)
     fuBANNER "SENSOR"
     ln -s /opt/tpot/etc/compose/sensor.yml $myTPOTCOMPOSE
@@ -807,28 +812,29 @@ mkdir -vp /data/adbhoney/{downloads,log} \
          /data/conpot/log \
          /data/citrixhoneypot/logs \
          /data/cowrie/{downloads,keys,misc,log,log/tty} \
-	       /data/ddospot/{bl,db,log} \
+         /data/ddospot/{bl,db,log} \
-	       /data/dicompot/{images,log} \
+         /data/dicompot/{images,log} \
          /data/dionaea/{log,bistreams,binaries,rtp,roots,roots/ftp,roots/tftp,roots/www,roots/upnp} \
          /data/elasticpot/log \
          /data/elk/{data,log} \
-	       /data/endlessh/log \
+         /data/endlessh/log \
          /data/fatt/log \
          /data/honeytrap/{log,attacks,downloads} \
          /data/glutton/log \
-	       /data/hellpot/log \
+         /data/hellpot/log \
          /data/heralding/log \
          /data/honeypots/log \
          /data/honeypy/log \
          /data/honeysap/log \
-	       /data/ipphoney/log \
+         /data/ipphoney/log \
+         /data/log4pot/{log,payloads} \
          /data/mailoney/log \
          /data/medpot/log \
          /data/nginx/{log,heimdall} \
          /data/emobility/log \
          /data/ews/conf \
          /data/rdpy/log \
-	       /data/redishoneypot/log \
+         /data/redishoneypot/log \
          /data/spiderfoot \
          /data/suricata/log \
          /data/tanner/{log,files} \

update.sh

@@ -218,8 +218,13 @@ echo
 
 # Ensure some defaults
 echo "### Ensure some T-Pot defaults with regard to some folders, permissions and configs."
-sed -i '/^port/Id' /etc/ssh/sshd_config
+sed -i '/^port/I,$d' /etc/ssh/sshd_config
-echo "Port 64295" >> /etc/ssh/sshd_config
+tee -a /etc/ssh/sshd_config << EOF
+Port 64295
+Match Group tpotlogs
+        PermitOpen 127.0.0.1:64305
+        ForceCommand /usr/bin/false
+EOF
 echo
 
 ### Ensure creation of T-Pot related folders, just in case
@@ -243,6 +248,8 @@ mkdir -vp /data/adbhoney/{downloads,log} \
          /data/honeypy/log \
          /data/honeysap/log \
          /data/ipphoney/log \
+         /data/log4pot/{log,payloads} \
+         /data/log4pot/log \
          /data/mailoney/log \
          /data/medpot/log \
          /data/nginx/{log,heimdall} \