Prepare for merge into master

CHANGELOG.md

@@ -1,45 +1,36 @@
 # Release Notes / Changelog
-T-Pot 22.04.0 is probably the most feature rich release ever provided with long awaited (wanted!) features readily available after installation. 
+T-Pot 24.04.0 marks probably the largest change in the history of the project. While most of the changes have been made to the underlying platform some changes will be standing out in particular - a T-Pot ISO image will no longer be provided with the benefit that T-Pot will now run on multiple Linux distributions (Alma Linux, Debian, Fedora, OpenSuse, Raspbian, Rocky Linux, Ubuntu), Raspberry Pi (optimized) and macOS / Windows (limited).
 
 ## New Features
-* **Distributed** Installation with **HIVE** and **HIVE_SENSOR**
+* **Distributed** Installation is now using NGINX reverse proxy instead of SSH to transmit **HIVE_SENSOR** logs to **HIVE**
-* **ARM64** support for all provided Docker images
+* **`deploy.sh`**, will make the deployment of sensor much easier and will automatically take care of the configuration. You only have to install the T-Pot sensor.  
-* **GeoIP Attack Map** visualizing Live Attacks on a dedicated webpage
+* **T-Pot Init** is the foundation for running T-Pot on multiple Linux distributions and will also ensure to restart containers with failed healthchecks using **autoheal**
-* **Kibana Live Attack Map** visualizing Live Attacks from different **HIVE_SENSORS**
+* **T-Pot Installer** is now mostly Ansible based providing a universal playbook for the most common Linux distributions 
-* **Blackhole** is a script trying to avoid mass scanner detection 
+* **T-Pot Uninstaller** allows to uninstall T-Pot, while not recommended for general usage, this comes in handy for testing purposes
-* **Elasticvue** a web front end for browsing and interacting with an Elastic Search cluster
+* **T-Pot Customizer (`compose/customizer.py`)** is here to assist you in the creation of a customized `docker-compose.yml`
-* **Ddospot** a honeypot for tracking and monitoring UDP-based Distributed Denial of Service (DDoS) attacks
+* **T-Pot Landing Page** has been redesigned and simplified
-* **Endlessh** is a SSH tarpit that very slowly sends an endless, random SSH banner
+![T-Pot-WebUI](doc/tpotwebui.png)
-* **HellPot** is an endless honeypot based on Heffalump that sends unruly HTTP bots to hell
+* **Kibana Dashboards, Objects** fully refreshed in favor of Lens based objects
-* **qHoneypots** 25 honeypots in a single container for monitoring network traffic, bots activities, and username \ password credentials
+![Dashbaord](doc/kibana_a.png)
-* **Redishoneypot** is a honeypot mimicking some of the Redis' functions
+* **Wordpot** is added as new addition to the available honeypots within T-Pot and will run on `tcp/8080` by default.
-* **SentryPeer** a dedicated SIP honeypot
+* **Raspberry Pi** is now supported using a dedicated `mobile.yml` (why this is called mobile will be revealed soon!)
-* **Index Lifecycle Management** for Elasticseach indices is now being used
+* **GeoIP Attack Map** is now aware of connects / disconnects and thus eliminating required reloads
-
+* **Docker**, where possible, will now be installed directly from the Docker repositories to avoid any incompatibilities
-## Upgrades
+* **`.env`** now provides a single configuration file for the T-Pot related settings
-* **Debian 11.x** is now being used for the T-Pot ISO images and required for post installs
+* **`genuser.sh`** can now be used to add new users to the T-Pot Landing Page as part of the T-Pot configuration file (`.env`)
-* **Elastic Stack 8.x** is now provided as Docker images
 
 ## Updates
-* **Honeypots** and **tools** were updated to their latest masters and releases
+* **Honeypots** and **tools** were updated to their latest pushed code and / or releases
+* Where possible Docker Images will now use Alpine 3.19
 * Updates will be provided continuously through Docker Images updates 
 
 ## Breaking Changes
-* For security reasons all Py2.x honeypots with the need of PyPi packages have been removed: **HoneyPy**, **HoneySAP** and **RDPY**
+* There is no option to migrate a previous installation to T-Pot 24.04.0, you can try to transfer the old `data` folder to the new T-Pot installation, but a working environment depends on too many other factors outside of our control and a new installation is simply faster.
-* If you are upgrading from a previous version of T-Pot (20.06.x) you need to import the new Kibana objects or some of the functionality will be broken or will be unavailabe
+* Most of the support scripts were moved into the **T-Pot Init** image and are no longer available directly on the host.
-* **Cyberchef** is now part of the Nginx Docker image, no longer as individual image
+* Cockpit is no longer available as part of T-Pot itself. However, where supported, you can simply install the `cockpit` package.
-* **ElasticSearch Head** is superseded by **Elasticvue** and part the Nginx Docker image
-* **Heimdall** is no longer supported and superseded with a new Bento based landing page
-* **Elasticsearch Curator** is no longer supprted and superseded with **Index Lifecycle Policies** available through Kibana.
 
 # Thanks & Credits
-* @ghenry, for some fun late night debugging and of course SentryPeer!
-* @giga-a, for adding much appreciated features (i.e. JSON logging, 
-X-Forwarded-For, etc.) and of course qHoneypots! 
 * @sp3t3rs, @trixam, for their backend and ews support!
-* @tadashi-oya, for spotting some errors and propose fixes!
+* @shark4ce for taking the time to test, debug and offer a solution #1472.
-* @tmariuss, @shaderecker for their cloud contributions!
-* @vorband, for much appreciated and helpful insights regarding the GeoIP Attack Map!
-* @yunginnanet, on not giving up on squashing a bug and of course Hellpot!
 
 ... and many others from the T-Pot community by opening valued issues and discussions, suggesting ideas and thus helping to improve T-Pot!

README.md

@@ -12,7 +12,7 @@ T-Pot is the all in one, optionally distributed, multiarch (amd64, arm64) honeyp
 4. Install `curl`: `$ sudo [apt, dnf, zypper] install curl` if not installed already
 5. Run installer as non-root from `$HOME`:
 ```
-env bash -c "$(curl -sL https://github.com/telekom-security/tpotce/raw/24.04/install.sh)"
+env bash -c "$(curl -sL https://github.com/telekom-security/tpotce/raw/master/install.sh)"
 ```
    * Follow instructions, read messages, check for possible port conflicts and reboot
 
@@ -125,6 +125,7 @@ T-Pot offers docker images for the following honeypots ...
 * [wordpot](https://github.com/gbrindisi/wordpot)
 
 ... alongside the following tools ...
+* [Autoheal](https://github.com/willfarrell/docker-autoheal) a tool to automatically restart containers with failed healthchecks.
 * [Cyberchef](https://gchq.github.io/CyberChef/) a web app for encryption, encoding, compression and data analysis.
 * [Elastic Stack](https://www.elastic.co/videos) to beautifully visualize all the events captured by T-Pot.
 * [Elasticvue](https://github.com/cars10/elasticvue/) a web front end for browsing and interacting with an Elasticsearch cluster.
@@ -326,9 +327,9 @@ Choose a supported distro of your choice. It is recommended to use the minimum /
 Sometimes it is just nice if you can spin up a T-Pot instance on macOS or Windows, i.e. for development, testing or just the fun of it. As Docker Desktop is rather limited not all honeypot types or T-Pot features are supported. Also remember, by default the macOS and Windows firewall are blocking access from remote, so testing is limited to the host. For production it is recommended to run T-Pot on [Linux](#choose-your-distro).<br>
 To get things up and running just follow these steps:
 1. Install Docker Desktop for [macOS](https://docs.docker.com/desktop/install/mac-install/) or [Windows](https://docs.docker.com/desktop/install/windows-install/).
-2. Clone the GitHub repository: `git clone https://github.com/telekom-security/tpotce -b 24.04`.
+2. Clone the GitHub repository: `git clone https://github.com/telekom-security/tpotce`
 3. Go to: `cd ~/tpotce`
-4. Copy `cp compose/mac_win.yml ./docker-compose.yml`.
+4. Copy `cp compose/mac_win.yml ./docker-compose.yml`
 5. Create a `WEB_USER` by running `~/tpotce/genuser.sh`
 6. Adjust the `.env` file by changing `TPOT_OSTYPE=linux` to either `mac` or `win`:
    ```
@@ -575,6 +576,9 @@ sudo su -
 docker login
 ```
 
+### **T-Pot Networking Fails**
+T-Pot is designed to only run on machines with a single NIC. T-Pot will try to grab the interface with the default route, however it is not guaranteed that this will always succeed. At best use T-Pot on machines with only a single NIC.
+
 ## Start T-Pot
 The T-Pot service automatically starts and stops on each reboot (which occurs once on a daily basis as setup in `sudo crontab -l` during installation).
 <br>
@@ -705,7 +709,7 @@ The software that T-Pot is built on uses the following licenses.
 <br>GPLv2: [conpot](https://github.com/mushorg/conpot/blob/master/LICENSE.txt), [dionaea](https://github.com/DinoTools/dionaea/blob/master/LICENSE), [honeytrap](https://github.com/armedpot/honeytrap/blob/master/LICENSE), [suricata](http://suricata-ids.org/about/open-source/)
 <br>GPLv3: [adbhoney](https://github.com/huuck/ADBHoney), [elasticpot](https://gitlab.com/bontchev/elasticpot/-/blob/master/LICENSE), [ewsposter](https://github.com/telekom-security/ews/), [log4pot](https://github.com/thomaspatzke/Log4Pot/blob/master/LICENSE), [fatt](https://github.com/0x4D31/fatt/blob/master/LICENSE), [heralding](https://github.com/johnnykv/heralding/blob/master/LICENSE.txt), [ipphoney](https://gitlab.com/bontchev/ipphoney/-/blob/master/LICENSE), [redishoneypot](https://github.com/cypwnpwnsocute/RedisHoneyPot/blob/main/LICENSE), [sentrypeer](https://github.com/SentryPeer/SentryPeer/blob/main/LICENSE.GPL-3.0-only), [snare](https://github.com/mushorg/snare/blob/master/LICENSE), [tanner](https://github.com/mushorg/snare/blob/master/LICENSE)
 <br>Apache 2 License: [cyberchef](https://github.com/gchq/CyberChef/blob/master/LICENSE), [dicompot](https://github.com/nsmfoo/dicompot/blob/master/LICENSE), [elasticsearch](https://github.com/elasticsearch/elasticsearch/blob/master/LICENSE.txt), [logstash](https://github.com/elasticsearch/logstash/blob/master/LICENSE), [kibana](https://github.com/elasticsearch/kibana/blob/master/LICENSE.md), [docker](https://github.com/docker/docker/blob/master/LICENSE)
-<br>MIT license: [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot/blob/master/LICENSE), [ddospot](https://github.com/aelth/ddospot/blob/master/LICENSE), [elasticvue](https://github.com/cars10/elasticvue/blob/master/LICENSE), [glutton](https://github.com/mushorg/glutton/blob/master/LICENSE), [hellpot](https://github.com/yunginnanet/HellPot/blob/master/LICENSE), [maltrail](https://github.com/stamparm/maltrail/blob/master/LICENSE)
+<br>MIT license: [autoheal](https://github.com/willfarrell/docker-autoheal?tab=MIT-1-ov-file#readme), [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot/blob/master/LICENSE), [ddospot](https://github.com/aelth/ddospot/blob/master/LICENSE), [elasticvue](https://github.com/cars10/elasticvue/blob/master/LICENSE), [glutton](https://github.com/mushorg/glutton/blob/master/LICENSE), [hellpot](https://github.com/yunginnanet/HellPot/blob/master/LICENSE), [maltrail](https://github.com/stamparm/maltrail/blob/master/LICENSE)
 <br> Unlicense: [endlessh](https://github.com/skeeto/endlessh/blob/master/UNLICENSE)
 <br> Other: [citrixhoneypot](https://github.com/MalwareTech/CitrixHoneypot#licencing-agreement-malwaretech-public-licence), [cowrie](https://github.com/cowrie/cowrie/blob/master/LICENSE.rst), [mailoney](https://github.com/awhitehatter/mailoney), [Elastic License](https://www.elastic.co/licensing/elastic-license), [Wordpot](https://github.com/gbrindisi/wordpot)
 <br> AGPL-3.0: [honeypots](https://github.com/qeeqbox/honeypots/blob/main/LICENSE)

SECURITY.md

@@ -3,8 +3,8 @@
 ## Supported Versions
 
 | Version | Supported          |
-|---------|--------------------|
+|-------|--------------------|
-| 24.04.x | :white_check_mark: |
+| 24.04 | :white_check_mark: |
 
 
 ## Reporting a Vulnerability

install.sh

@@ -119,7 +119,7 @@ fi
 if [ ! -f installer/install/tpot.yml ] && [ ! -f tpot.yml ];
   then
     echo "### Now downloading T-Pot Ansible Installation Playbook ... "
-    wget -qO tpot.yml https://github.com/telekom-security/tpotce/raw/24.04/installer/install/tpot.yml
+    wget -qO tpot.yml https://github.com/telekom-security/tpotce/raw/master/installer/install/tpot.yml
     myANSIBLE_TPOT_PLAYBOOK="tpot.yml"
     echo
   else

installer/install/tpot.yml

@@ -694,7 +694,7 @@
       git:
         repo: 'https://github.com/telekom-security/tpotce'
         dest: '/home/{{ ansible_user_id }}/tpotce/'
-        version: 24.04
+        version: master
         clone: yes
         update: no
       when: ansible_distribution in ["AlmaLinux", "Debian", "Fedora", "openSUSE Tumbleweed", "Raspbian", "Rocky", "Ubuntu"]

update.sh

@@ -61,7 +61,7 @@ function fuSELFUPDATE () {
 	    return
 	fi
 	### DEV
-	myRESULT=$(git diff --name-only origin/24.04 | grep "^update.sh")
+	myRESULT=$(git diff --name-only origin/master | grep "^update.sh")
 	if [ "$myRESULT" == "update.sh" ];
 	  then
 	    echo "###### $myBLUE""Found newer version, will be pulling updates and restart myself.""$myWHITE"