tpotce/CHANGELOG.md

# Changelog

## 20210219
- **Rebuild Snare, Tanner, Redis, Phpox**
  - Rebuild images to their latest masters and upgrade Alpine OS to 3.13 where possible.
- **Bump Elastic Stack to 7.11.1**
  - Updgrade Elastic Stack Images to 7.11.1 and update License Info to reflect new Elastic License.
  - Prepare for new release.

## 20210218
- **Rebuild Conpot, EWSPoster, Cowrie, Glutton, Dionaea**
  - Rebuild images to their latest masters and upgrade Alpine OS to 3.13 where possible.

## 20210216
- **Bump Heralding to 1.0.7**
  - Rebuild and upgrade image to 1.0.7 and upgrade Alpine OS to 3.13.
  - Enable SMTPS for Heralding.
- **Rebuild IPPHoney, Fatt, EWSPoster, Spiderfoot**
  - Rebuild images to their latest masters and upgrade Alpine OS to 3.13 where possible.
  - Upgrade Spiderfoot to 3.3

## 20210215
- **Rebuild Dicompot, p0f, Medpot, Honeysap, Heimdall, Elasticpot, Citrixhoneypot, Ciscoasa**
  - Rebuild images to their latest masters and upgrade Alpine OS to 3.13 where possible.

## 20210212
- **Rebuild Cyberchef, Adbhoney, Elastic Stack**
  - Rebuild images to their latest masters and upgrade Alpine OS to 3.13 where possible.
  - Bump Elastic Stack to 7.11.0
  - Bump Cyberchef to 9.27.0

## 20210119
- **Bump Dionaea to 0.11.0**
  - Upgrade Dionaea to 0.11.0, rebuild image and upgrade Alpine OS to 3.13.

## 20210106
- **Update Internet IF retrieval**
  - To be consistent with @adepasquale PR #746 fatt, glutton and p0f Dockerfiles were updated accordingly.
  - Merge PR #746 from @adepasquale, thank you!

## 20201228
- **Fix broken SQlite DB**
  - Fix a broken `app.sqlite` in Heimdall
- **Avoid ghcr.io because of slow transfers**
- **Remove netselect-apt**
  - causes too many unpredictable errors #733 as the latest example

## 20201210
- **Bump Elastic Stack 7.10.1, EWSPoster to 1.12**

## 20201202
- **Update Elastic Stack to 7.10.0**

## 20201130
- **Suricata, use suricata-update for rule management**
  - As a bonus we can now run "suricata-update" using docker-exec, triggering both a rule update and a Suricata rule reload.
  - Thanks to @adepasquale!

## 20201126
- **Suricata, update suricata.yaml for 6.x**
  - Merge in the latest updates from suricata-6.0.x while at the same time keeping the custom T-Pot configuration.
  - Thanks to @adepasquale!
- **Bump Cowrie to 2.2.0**

## 20201028
- **Bump Suricata to 5.0.4, Spiderfoot to 3.2.1, Dionaea to 0.9.2, IPPHoney, Heralding, Conpot to latest masters**

## 20201027
- **Bump Dicompot to latest master, Elastic Stack to 7.9.3**

## 20201005
- **Bump Elastic Stack to 7.9.2**
  - @brianlechthaler, thanks for PR #706, which had issues regarding Elastic Stack and resulted in reverting to 7.9.1

## 20200904
- **Release T-Pot 20.06.1**
  - Github offers a free Docker Container Registry for public packages. For our Open Source projects we want to make sure to have everything in one place and thus moving from Docker Hub to the GitHub Container Registry.
- **Bump Elastic Stack**
  - Update the Elastic Stack to 7.9.1.
- **Rebuild Images**
  - All docker images were rebuilt based on the latest (and stable running) versions of the tools and honeypots and have been pinned to specific Alpine / Debian versions and git commits so rebuilds will less likely fail.
- **Cleaning up**
  - Clean up old references and links.

## 20200630
- **Release T-Pot 20.06**
  - After 4 months of public testing with the NextGen edition T-Pot 20.06 can finally be released.
- **Debian Buster**
  - With the release of Debian Buster T-Pot now has access to all packages required right out of the box.
- **Add new honeypots**
  - [Dicompot](https://github.com/nsmfoo/dicompot) by @nsmfoo is a low interaction honeypot for the Dicom protocol which is the international standard to process medical imaging information. Together with Medpot which supports the HL7 protocol T-Pot is now offering a Medical Installation type.
  - [Honeysap](https://github.com/SecureAuthCorp/HoneySAP) by SecureAuthCorp is a low interaction honeypot for the SAP services, in case of T-Pot configured for the SAP router.
  - [Elasticpot](https://gitlab.com/bontchev/elasticpot) by Vesselin Bontchev replaces ElasticpotPY as a low interaction honeypot for Elasticsearch with more features, plugins and scripted responses.
- **Rebuild Images**
  - All docker images were rebuilt based on the latest (and stable running) versions of the tools and honeypots. Mostly the images now run on Alpine 3.12 / Debian Buster. However some honeypots / tools still reuire Alpine 3.11 / 3.10 to run properly.
- **Install Types**
  - All docker-compose files (`/opt/tpot/etc/compose`) were remixed and most of the NextGen honeypots are now available in Standard.
  - There is now a **Medical** Installation Type with Dicompot and Medpot which will be of most interest for medical institutions to get started with T-Pot.
- **Update Tools**
  - Connecting to T-Pot via `https://<ip>:64297` brings you to the T-Pot Landing Page now which is based on Heimdall and the latest NGINX enforcing TLS 1.3.
  - The ELK stack was updated to 7.8.0 and stripped down to the necessary core functions (where possible) for T-Pot while keeping ELK RAM requirements to a minimum (8GB of RAM is recommended now). The number of index pattern fields was reduced to **697** which increases performance significantly. There are **22** Kibana Dashboards, **397** Kibana Visualizations and **24** Kibana Searches readily available to cover all your needs to get started and familiar with T-Pot.
  - Cyberchef was updated to 9.21.0.
  - Elasticsearch Head was updated to the latest version available on GitHub.
  - Spiderfoot was updated to latest 3.1 dev.
- **Landing Page**
  - After logging into T-Pot via web you are now greeted with a beautifully designed landing page.
- **Countless Tweaks and improvements**
  - Under the hood lots of tiny tweaks, improvements and a few bugfixes will increase your overall experience with T-Pot.

## 20200316
- **Move from Sid to Stable**
  - Debian Stable has now all the packages and versions we need for T-Pot. As a consequence we can now move to the `stable` branch.

## 20200310
- **Add 2FA to Cockpit**
  - Just run `2fa.sh` to enable two factor authentication in Cockpit.
- **Find fastest mirror with netselect-apt**
  - Netselect-apt will find the fastest mirror close to you (outgoing ICMP required).

## 20200309
- **Bump Nextgen to 20.06**
  - All NextGen images have been rebuilt to their latest master.
  - ElasticStack bumped to 7.6.1 (Elasticsearch will need at least 2048MB of RAM now, T-Pot at least 8GB of RAM) and tweak to accomodate changes of 7.x.
  - Fixed errors in Tanner / Snare which will now handle downloads of malware via SSL and store them correctly (thanks to @afeena).
  - Fixed errors in Heralding which will now improve on RDP connections (thanks to @johnnykv, @realsdx).
  - Fixed error in honeytrap which will now build in Debian/Buster (thanks to @tillmannw).
  - Mailoney is now logging in JSON format (thanks to @monsherko).
  - Base T-Pot landing page on Heimdall.
  - Tweaking of tools and some minor bug fixing

## 20200116
- **Bump ELK to latest 6.8.6**
- **Update ISO image to fix upstream bug of missing kernel modules**
- **Include dashboards for CitrixHoneypot**
  - Please run `/opt/tpot/update.sh` for the necessary modifications, omit the reboot and run `/opt/tpot/bin/tped.sh` to (re-)select the NextGen installation type.
  - This update requires the latest Kibana objects as well. Download the latest from https://raw.githubusercontent.com/telekom-security/tpotce/master/etc/objects/kibana_export.json.zip, unzip and import the objects within Kibana WebUI > Management > Saved Objects > Export / Import". All objects will be overwritten upon import, make sure to run an export first.

## 20200115
- **Prepare integration of CitrixHoneypot**
  - Prepare integration of [CitrixHoneypot](https://github.com/MalwareTech/CitrixHoneypot) by MalwareTech
  - Integration into ELK is still open
  - Please run `/opt/tpot/update.sh` for the necessary modifications, omit the reboot and run `/opt/tpot/bin/tped.sh` to (re-)select the NextGen installation type.

## 20191224
- **Use pigz, optimize logrotate.conf**
  - Use `pigz` for faster archiving, especially with regard to high volumes of logs - Thanks to @workandresearchgithub!
  - Optimize `logrotate.conf` to improve archiving speed and get rid of multiple compression, also introduce `pigz`.

## 20191121
- **Bump ADBHoney to latest master**
  - Use latest version of ADBHoney, which now fully support Python 3.x - Thanks to @huuck!

## 20191113, 20191104, 20191103, 20191028
- **Switch to Debian 10 on OTC, Ansible Improvements**
  - OTC now supporting Debian 10 - Thanks to @shaderecker!

## 20191028
- **Fix an issue with pip3, yq**
  - `yq` needs rehashing.

## 20191026
- **Remove cockpit-pcp**
  - `cockpit-pcp` floods swap for some reason - removing for now.

## 20191022
- **Bump Suricata to 5.0.0**

## 20191021
- **Bump Cowrie to 2.0.0**

## 20191016
- **Tweak installer, pip3, Heralding**
  - Install `cockpit-pcp` right from the start for machine monitoring in cockpit.
  - Move installer and update script to use pip3.
  - Bump heralding to latest master (1.0.6) - Thanks @johnnykv!

## 20191015
- **Tweaking, Bump glutton, unlock ES script**
  - Add `unlock.sh` to unlock ES indices in case of lockdown after disk quota has been reached.
  - Prevent too much terminal logging from p0f and glutton since `daemon.log` was filled up.
  - Bump glutton to latest master now supporting payload_hex. Thanks to @glaslos.

## 20191002
- **Merge**
  - Support Debian Buster images for AWS #454
  - Thank you @piffey

## 20190924
- **Bump EWSPoster**
  - Supports Python 3.x
  - Thank you @Trixam

## 20190919
- **Merge**
  - Handle non-interactive shells #454
  - Thank you @Oogy

## 20190907
- **Logo tweaking**
  - Add QR logo

## 20190828
- **Upgrades and rebuilds**
  - Bump Medpot, Nginx and Adbhoney to latest master
  - Bump ELK stack to 6.8.2
  - Rebuild Mailoney, Honeytrap, Elasticpot and Ciscoasa
  - Add 1080p T-Pot wallpaper for download

## 20190824
- **Add some logo work**
  - Thanks to @thehadilps's suggestion adjusted social preview
  - Added 4k T-Pot wallpaper for download

## 20190823
- **Fix for broken Fuse package**
  - Fuse package in upstream is broken
  - Adjust installer as workaround, fixes #442

## 20190816
- **Upgrades and rebuilds**
  - Adjust Dionaea to avoid nmap detection, fixes #435 (thanks @iukea1)
  - Bump Tanner, Cyberchef, Spiderfoot and ES Head to latest master

## 20190815
- **Bump ELK stack to 6.7.2**
  - Transition to 7.x must iterate slowly through previous versions to prevent changes breaking T-Pots

## 20190814
- **Logstash Translation Maps improvement**
  - Download translation maps rather than running a git pull
  - Translation maps will now be bzip2 compressed to reduce traffic to a minimum
  - Fixes #432

## 20190802
- **Add support for Buster as base image**
  - Install ISO is now based on Debian Buster
  - Installation upon Debian Buster is now supported

## 20190701
- **Reworked Ansible T-Pot Deployment**
  - Transitioned from bash script to all Ansible
  - Reusable Ansible Playbook for OpenStack clouds
  - Example Showcase with our Open Telekom Cloud
  - Adaptable for other cloud providers

## 20190626
- **HPFEEDS Opt-In commandline option**
  - Pass a hpfeeds config file as a commandline argument
  - hpfeeds config is saved in `/data/ews/conf/hpfeeds.cfg`
  - Update script restores hpfeeds config

## 20190604
- **Finalize Fatt support**
  - Build visualizations, searches, dashboards
  - Rebuild index patterns
  - Some finishing touches

## 20190601
- **Start supporting Fatt, remove Glastopf**
  - Build Dockerfile, Adjust logstash, installer, update and such.
  - Glastopf is no longer supported within T-Pot

## 20190528+20190531
- **Increase total number of fields**
  - Adjust total number of fileds for logstash templae from 1000 to 2000.

## 20190526
- **Fix build for Cowrie**
  - Upstream changes required a new package `py-bcrypt`.

## 20190525
- **Fix build for RDPY**
  - Building was prevented due to cache error which occurs lately on Alpine if `apk` is using `--no-ache' as options.

## 20190520
- **Adjust permissions for /data folder**
  - Now it is possible to download files from `/data` using SCP, WINSCP or CyberDuck.

## 20190513
- **Added Ansible T-Pot Deployment on Open Telekom Cloud**
  - Reusable Ansible Playbooks for all cloud providers
  - Example Showcase with our Open Telekom Cloud

## 20190511
- **Add hptest script**
  - Quickly test if the honeypots are working with `hptest.sh <[ip,host]>` based on nmap.

## 20190508
- **Add tsec / install user to tpot group**
  - For users being able to easily download logs from the /data folder the installer now adds the `tpot` or the logged in user (`who am i`) via `usermod -a -G tpot <user>` to the tpot group. Also /data permissions will now be enforced to `770`, which is necessary for directory listings.

## 20190502
- **Fix KVPs**
  - Some KVPs for Cowrie changed and the tagcloud was not showing any values in the Cowrie dashboard.
  - New installations are not affected, however existing installations need to import the objects from /opt/tpot/etc/objects/kibana-objects.json.zip.
- **Makeiso**
  - Move to Xorriso for building the ISO image.
  - This allows to support most of the Debian based distros, i.e. Debian, MxLinux and Ubuntu.

## 20190428
- **Rebuild ISO**
  - The install ISO needed a rebuilt after some changes in the Debian mirrors.
- **Disable Netselect**
  - After some reports in the issues that some Debian mirrors were not fully synced and thus some packages were unavailable the netselect-apt feature was disabled.

## 20190406
- **Fix for SSH**
  - In some situations the SSH Port was not written to a new line (thanks to @dpisano for reporting).
- **Fix race condition for apt-fast**
  - Curl and wget need to be installed before apt-fast installation.

## 20190404
- **Fix #332**
  - If T-Pot, opposed to the requirements, does not have full internet access netselect-apt fails to determine the fastest mirror as it needs ICMP and UDP outgoing. Should netselect-apt fail the default mirrors will be used.
- **Improve install speed with apt-fast**
  - Migrating from a stable base install to Debian (Sid) requires downloading lots of packages. Depending on your geo location the download speed was already improved by introducing netselect-apt to determine the fastest mirror. With apt-fast the downloads will be even faster by downloading packages not only in parallel but also with multiple connections per package.

`git log --date=format:"## %Y%m%d" --pretty=format:"%ad %n- **%s**%n  - %b"`