Merge pull request #262 from dtag-dev-sec/18.04

Prepare T-Pot 18.11 Release

CONTRIBUTING.MD

@@ -6,18 +6,11 @@ Thank you for your decision to contribute to T-Pot.
 
 Please feel free to post your problems, ideas and issues [here](https://github.com/dtag-dev-sec/tpotce/issues). We will try to answer ASAP, but to speed things up we encourage you to ...
 - [ ] Use the [search function](https://github.com/dtag-dev-sec/tpotce/issues?utf8=%E2%9C%93&q=) first
-- [ ] Check the [FAQ](#faq)
+- [ ] Check the FAQs in our [WIKI](https://github.com/dtag-dev-sec/tpotce/wiki)
 - [ ] Provide [basic support information](#info) with regard to your issue
 
 Thank you :smiley:
 
--
-
-<a name="faq"></a>
-### FAQ
-
-##### Where can I find the honeypot logs?
-###### The honeypot logs are located in `/data/`. You have to login via ssh and run `sudo su -` and then `cd /data/`. Do not change any permissions here or T-Pot will fail to work.
 
 -
 
@@ -25,13 +18,13 @@ Thank you :smiley:
 <a name="info"></a>
 ### Basic support information
 
-- What T-Pot version are you currtently using?
+- What T-Pot version are you currently using?
 - Are you running on a Intel NUC or a VM?
 - How long has your installation been running?
 - Did you install any upgrades or packages?
 - Did you modify any scripts?
 - Have you turned persistence on/off?
-- How much RAM available (login via ssh and run `htop`)?
+- How much RAM is available (login via ssh and run `htop`)?
 - How much stress are the CPUs under (login via ssh and run `htop`)?
 - How much swap space is being used (login via ssh and run `htop`)?
 - How much free disk space is available (login via ssh and run `sudo df -h`)?

README.md

@@ -1,55 +1,42 @@
-# T-Pot 17.10
+# T-Pot 18.11
 
-This repository contains the necessary files to create the **[T-Pot](https://github.com/dtag-dev-sec/tpotce/releases)** ISO image.
-The image can then be used to install T-Pot on a physical or virtual machine.
-
-In October 2016 we released
-[T-Pot 16.10](http://dtag-dev-sec.github.io/mediator/feature/2016/10/31/t-pot-16.10.html)
-
-# T-Pot 17.10
-
-T-Pot 17.10 runs on the latest 16.04 LTS Ubuntu Server Network Installer image, is based on
+T-Pot 18.11 runs on the latest 18.04.x LTS Ubuntu Server Network Installer image, is based on
 
 [docker](https://www.docker.com/), [docker-compose](https://docs.docker.com/compose/)
 
 and includes dockerized versions of the following honeypots
 
+* [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot),
 * [conpot](http://conpot.org/),
 * [cowrie](http://www.micheloosterhof.com/cowrie/),
 * [dionaea](https://github.com/DinoTools/dionaea),
 * [elasticpot](https://github.com/schmalle/ElasticPot),
-* [emobility](https://github.com/dtag-dev-sec/emobility),
-* [glastopf](http://glastopf.org/),
+* [glastopf](http://mushmush.org/),
+* [glutton](https://github.com/mushorg/glutton),
+* [heralding](https://github.com/johnnykv/heralding),
 * [honeytrap](https://github.com/armedpot/honeytrap/),
 * [mailoney](https://github.com/awhitehatter/mailoney),
-* [rdpy](https://github.com/citronneur/rdpy) and
-* [vnclowpot](https://github.com/magisterquis/vnclowpot)
+* [medpot](https://github.com/schmalle/medpot),
+* [rdpy](https://github.com/citronneur/rdpy),
+* [snare](http://mushmush.org/),
+* [tanner](http://mushmush.org/)
 
 
 Furthermore we use the following tools
 
+* [Cockpit](https://cockpit-project.org/running) for a lightweight, webui for docker, os, real-time performance monitoring and web terminal.
+* [Cyberchef](https://gchq.github.io/CyberChef/) a web app for encryption, encoding, compression and data analysis.
 * [ELK stack](https://www.elastic.co/videos) to beautifully visualize all the events captured by T-Pot.
 * [Elasticsearch Head](https://mobz.github.io/elasticsearch-head/) a web front end for browsing and interacting with an Elastic Search cluster.
-* [Netdata](http://my-netdata.io/) for real-time performance monitoring.
-* [Portainer](http://portainer.io/) a web based UI for docker.
 * [Spiderfoot](https://github.com/smicallef/spiderfoot) a open source intelligence automation tool.
 * [Suricata](http://suricata-ids.org/) a Network Security Monitoring engine.
-* [Wetty](https://github.com/krishnasrinivas/wetty) a web based SSH client.
-
 
 
 # TL;DR
-1. Meet the [system requirements](#requirements). The T-Pot installation needs at least 4 GB RAM and 64 GB free disk space as well as a working internet connection.
+1. Meet the [system requirements](#requirements). The T-Pot installation needs at least 6-8 GB RAM and 128 GB free disk space as well as a working internet connection.
 2. Download the T-Pot ISO from [GitHub](https://github.com/dtag-dev-sec/tpotce/releases) or [create it yourself](#createiso).
 3. Install the system in a [VM](#vm) or on [physical hardware](#hw) with [internet access](#placement).
-4. Enjoy your favorite beverage - [watch](http://sicherheitstacho.eu/?peers=communityPeers) and [analyze](#kibana).
-
-# T-Pot-Autoinstaller
-T-Pot may also be installed on an existing machine using the [T-Pot-Autoinstaller](https://github.com/dtag-dev-sec/t-pot-autoinstall).
-
-# Seeing is believing :bowtie:
-
-[![T-Pot 17.10](https://img.youtube.com/vi/G-_OabDowFU/0.jpg)](https://youtu.be/G-_OabDowFU)
+4. Enjoy your favorite beverage - [watch](https://sicherheitstacho.eu) and [analyze](#kibana).
 
 
 # Table of Contents
@@ -61,8 +48,11 @@ T-Pot may also be installed on an existing machine using the [T-Pot-Autoinstalle
   - [Create your own ISO Image](#createiso)
   - [Running in a VM](#vm)
   - [Running on Hardware](#hardware)
+  - [Post Install Manual](#postinstall)
+  - [Post Install Auto](#postinstallauto)
   - [First Run](#firstrun)
   - [System Placement](#placement)
+- [Updates](#updates)
 - [Options](#options)
   - [SSH and web access](#ssh)
   - [Kibana Dashboard](#kibana)
@@ -78,147 +68,155 @@ T-Pot may also be installed on an existing machine using the [T-Pot-Autoinstalle
 - [Stay tuned](#staytuned)
 - [Fun Fact](#funfact)
 
-<a name="background"></a>
+<a name="changelog"></a>
 # Changelog
-- **Size still matters** 😅
-	- All docker images have been rebuilt as micro containers based on Alpine Linux to even further reduce the image size and leading to image sizes (compressed) below the 50 MB mark. The uncompressed size of eMobility and the ELK stack could each be reduced by a whopping 600 MB!
-	- A "Everything" installation now takes roughly 1.6 GB download size
-- **docker-compose**
-	- T-Pot containers are now being controlled and monitored through docker-compose and a single configuration file `/opt/tpot/etc/tpot.yml` allowing for greater flexibility and resulting in easier image management (i.e. updated images).
-	- As a benefit only a single `systemd` script `/etc/systemd/system/tpot.service` is needed to start `systemctl start tpot` and stop `systemctl stop tpot` the T-Pot services.
-	- There are four pre-configured compose configurations which do reflect the T-Pot editions `/opt/tpot/etc/compose`. Simply stop the T-Pot services and copy i.e. `cp /opt/tpot/etc/compose/all.yml /opt/tpot/etc/tpot.yml`, restart the T-Pot services and the selcted edition will be running after downloading the required docker images.
-- **Introducing** [Spiderfoot](https://github.com/smicallef/spiderfoot) a open source intelligence automation tool.
-- **Installation** procedure simplified
-	- Within the Ubuntu Installer you only have to choose language settings
-	- After the first reboot the T-Pot installer checks if internet and required services are reachable before the installation procedure begins
-	- T-Pot Installer now uses a “dialog” which looks way better than the old text based installer
-	- `tsec` user & password dialog is now part of the T-Pot Installer
-	- The self-signed certificate is now created automatically to reduce unnecessary overhead for novice users
-	- New ASCII logo and login screen pointing to web and ssh logins
-	- Hostnames are now generated using an offline name generator, which still produces funny and collision free hostnames
-- **CVE IDs for Suricata**
-	- Our very own [Listbot](https://github.com/dtag-dev-sec/listbot) builds translation maps for Logstash. If Logstash registers a match the events' CVE ID will be stored alongside the event within Elasticsearch.
-- **IP Reputations**
-	- [Listbot](https://github.com/dtag-dev-sec/listbot) also builds translation maps for blacklisted IPs
-	- Based upon 30+ publicly available IP blacklisting sources listbot creates a logstash translation map matching the events' source IP addresses against the IPs reputation
-	- If the source IP is known to a blacklist service a corresponding tag will be stored with the event
-	- Updates occur on every logstash container start; by default every 24h
-- **Honeypot updates and improvements**
-	- All honeypots were updated to their latest & stable versions.
-- **New Honeypots** were added ...
-	* [mailoney](https://github.com/awhitehatter/mailoney)
-		- A low interaction SMTP honeypot
-	* [rdpy](https://github.com/citronneur/rdpy)
-		- A low interaction RDP honeypot
-	* [vnclowpot](https://github.com/magisterquis/vnclowpot)
-		- A low interaction VNC honeypot
-- **Persistence** is now enabled by default and will keep honeypot logs and tools data in `/data/` and its sub-folders by default for 30 days. You may change that behavior in `/opt/tpot/etc/logrotate/logrotate.conf`. ELK data however will be kept for 90 days by default. You may change that behavior in `/opt/tpot/etc/curator/actions.yml`. Scripts will be triggered through `/etc/crontab`.
-- **Updates**
-	- **Docker** was updated to the latest **1.12.6** release within Ubuntu 16.04.x LTS
-	- **ELK** was updated to the latest **Kibana 5.6.3**, **Elasticsearch 5.6.3** and **Logstash 5.6.3** releases.
-	- **Suricata** was updated to the latest **4.0.0** version including the latest **Emerging Threats** community ruleset.
+- **New honeypots**
+ - *Ciscoasa* a low interaction honeypot for the Cisco ASA component capable of detecting CVE-2018-0101, a DoS and remote code execution vulnerability.
+ - *Glutton* (NextGen) is the all eating honeypot
+ - *Heralding* a credentials catching honeypot.
+ - *Medpot* is a HL7 / FHIR honeypot.
+ - *Snare* is a web application honeypot sensor, is the successor of Glastopf. SNARE has feature parity with Glastopf and allows to convert existing web pages into attack surfaces.
+ - *Tanner* is SNARES' "brain". Every event is send from SNARE to TANNER, gets evaluated and TANNER decides how SNARE should respond to the client. This allows us to change the behaviour of many sensors on the fly. We are providing a TANNER instance for your use, but there is nothing stopping you from setting up your own instance.
+- **New tools**
+ - *Cockpit* is an interactive server admin interface. It is easy to use and very lightweight. Cockpit interacts directly with the operating system from a real Linux session in a browser.
+ - *Cyberchef* is the Cyber Swiss Army Knife - a web app for encryption, encoding, compression and data analysis.
+ - *grc* (commandline) is yet another colouriser (written in python) for beautifying your logfiles or output of commands.
+ - *multitail* (commandline) allows you to monitor logfiles and command output in multiple windows in a terminal, colorize, filter and merge.
+ - *tped.sh* (commandline) allows you to switch between T-Pot Editions after installation.
+- **Deprecated tools**
+ - *Netdata*, *Portainer* and *WeTTY* were superseded by *Cockpit* which is much more lightweight, perfectly well integrated into Ubuntu 18.04 LTS and of course comes with the same but a more basic feature set.
+- **New Standard Installation**
+ - The new standard installation is now running a whopping *14* honeypot instances.
+- **T-Pot Universal Installer**
+ - The T-Pot installer now also includes the option to install on a existing machine, the T-Pot-Autoinstaller is no longer necessary.
+- **Tighten Security**
+ - The docker containers are now running mostly with a read-only file system
+ - If possible using `setcap` to start daemons without root or dropping privileges
+ - Introducing `fail2ban` to ease up on `authorized_keys` requirement which is no longer necessary for `SSH`. Also to further prevent brute-force attacks on `Cockpit` and `NGINX` allowing for faster load times of the WebUI.
+- **Iptables exceptions for NFQ based honeypots**
+ - In previous versions `iptables`had manually be maintained, now a a script parses `/opt/tpot/etc/tpot.yml` and extracts port information to automatically generate exceptions for ports that should not be forwarded to NFQ.
+- **CI**
+ - The Kibana UI now uses a magenta theme.
+- **ES HEAD**
+ - A Java Script now automatically enters the correct FQDN / IP. A manual step is no longer required.
+- **ELK STACK**
+ - The ELK Stack was updated to the latest 6.x versions.
+ - This also means you can now expect the availability of basic *X-Pack-Feaures*, the full feature set however is only available to users with a valid license.
 - **Dashboards Makeover**
-	- We now have **160+ Visualizations** pre-configured and compiled to 14 individual **Kibana Dashboards** for every honeypot. Monitor all *honeypot events* locally on your T-Pot installation. Aside from *honeypot events* you can also view *Suricata NSM, Syslog and NGINX* events for a quick overview of local host events.
-	- View available IP reputation of any source IP address
-	- View available CVE ID for events
-	- More **Smart links** are now included.
+ - Because Kibana 6.x introduced so much whitespace the dashboards and some of the visualizations needed some overhaul. While it probably needs some getting used to the key was to focus on displaying as much information while not compromising on clarity.  
+ - Because of the new honeypots we now more than **200 Visualizations** pre-configured and compiled to 16 individual **Kibana Dashboards**. Monitor all *honeypot events* locally on your T-Pot installation. Aside from *honeypot events* you can also view *Suricata NSM and NGINX* events for a quick overview of wire events.
+- **Honeypot updates and improvements**
+ - All honeypots were updated to their latest stable versions.
+ - Docker images were mostly overhauled to tighten security even further
+ - Some of the honeypot configurations were modified to keep things fresh
 - **Update Feature**
-	- For the ones who like to live on the bleeding edge of T-Pot development there is now a update script available in `/opt/tpot/update.sh`. Just run the script and it will get the latest changes from the `master branch`. For now this feature is experimental and the first step to a true rolling release cycle.
-- **Files & Folders**
-	- While the `/data` folder is still in its old place, all T-Pot relevant files and folders have been restructured and will now be installed into `/opt/tpot`. Only a few system relevant files with regard to the installed OS and its services will be copied to locations outside the T-Pot base path.
+ - For the ones who like to live on the bleeding edge of T-Pot development there is now a update script available in `/opt/tpot/update.sh`.
+ - This feature is now in beta and is mostly intended to provide you with the latest development advances without the need of reinstalling T-Pot.
 
 <a name="concept"></a>
 # Technical Concept
 
-T-Pot is based on the network installer of Ubuntu Server 16.04.x LTS.
+T-Pot is based on the network installer of Ubuntu Server 18.04.x LTS.
 The honeypot daemons as well as other support components being used have been containerized using [docker](http://docker.io).
 This allows us to run multiple honeypot daemons on the same network interface while maintaining a small footprint and constrain each honeypot within its own environment.
 
-In T-Pot we combine the dockerized honeypots
-[conpot](http://conpot.org/),
-[cowrie](http://www.micheloosterhof.com/cowrie/),
-[dionaea](https://github.com/DinoTools/dionaea),
-[elasticpot](https://github.com/schmalle/ElasticPot),
-[emobility](https://github.com/dtag-dev-sec/emobility),
-[glastopf](http://glastopf.org/),
-[honeytrap](https://github.com/armedpot/honeytrap/),
-[mailoney](https://github.com/awhitehatter/mailoney),
-[rdpy](https://github.com/citronneur/rdpy) and
-[vnclowpot](https://github.com/magisterquis/vnclowpot) with
-[ELK stack](https://www.elastic.co/videos) to beautifully visualize all the events captured by T-Pot,
-[Elasticsearch Head](https://mobz.github.io/elasticsearch-head/) a web front end for browsing and interacting with an Elastic Search cluster,
-[Netdata](http://my-netdata.io/) for real-time performance monitoring,
-[Portainer](http://portainer.io/) a web based UI for docker,
-[Spiderfoot](https://github.com/smicallef/spiderfoot) a open source intelligence automation tool,
-[Suricata](http://suricata-ids.org/) a Network Security Monitoring engine and
-[Wetty](https://github.com/krishnasrinivas/wetty) a web based SSH client.
+In T-Pot we combine the dockerized honeypots ...
+* [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot),
+* [conpot](http://conpot.org/),
+* [cowrie](http://www.micheloosterhof.com/cowrie/),
+* [dionaea](https://github.com/DinoTools/dionaea),
+* [elasticpot](https://github.com/schmalle/ElasticPot),
+* [glastopf](http://mushmush.org/),
+* [glutton](https://github.com/mushorg/glutton),
+* [heralding](https://github.com/johnnykv/heralding),
+* [honeytrap](https://github.com/armedpot/honeytrap/),
+* [mailoney](https://github.com/awhitehatter/mailoney),
+* [medpot](https://github.com/schmalle/medpot),
+* [rdpy](https://github.com/citronneur/rdpy),
+* [snare](http://mushmush.org/),
+* [tanner](http://mushmush.org/)
 
-![Architecture](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/doc/architecture.png)
+... with the following tools ...
+* [Cockpit](https://cockpit-project.org/running) for a lightweight, webui for docker, os, real-time performance monitoring and web terminal.
+* [Cyberchef](https://gchq.github.io/CyberChef/) a web app for encryption, encoding, compression and data analysis.
+* [ELK stack](https://www.elastic.co/videos) to beautifully visualize all the events captured by T-Pot.
+* [Elasticsearch Head](https://mobz.github.io/elasticsearch-head/) a web front end for browsing and interacting with an Elastic Search cluster.
+* [Spiderfoot](https://github.com/smicallef/spiderfoot) a open source intelligence automation tool.
+* [Suricata](http://suricata-ids.org/) a Network Security Monitoring engine.
+
+... to give you the best out-of-the-box experience possible and a easy-to-use multi-honeypot appliance.
+
+![Architecture](doc/architecture.png)
 
 While data within docker containers is volatile we do now ensure a default 30 day persistence of all relevant honeypot and tool data in the well known `/data` folder and sub-folders. The persistence configuration may be adjusted in `/opt/tpot/etc/logrotate/logrotate.conf`. Once a docker container crashes, all other data produced within its environment is erased and a fresh instance is started from the corresponding docker image.<br>
 
 Basically, what happens when the system is booted up is the following:
 
 - start host system
-- start all the necessary services (i.e. docker-engine, reverse proxy, etc.)
-- start all docker containers via docker-compose (honeypots, nms, elk)
+- start all the necessary services (i.e. cockpit, docker, etc.)
+- start all docker containers via docker-compose (honeypots, nms, elk, etc.)
 
-Within the T-Pot project, we provide all the tools and documentation necessary to build your own honeypot system and contribute to our [community data view](http://sicherheitstacho.eu/?peers=communityPeers), a separate channel on our  [Sicherheitstacho](http://sicherheitstacho.eu) that is powered by T-Pot community data.
+Within the T-Pot project, we provide all the tools and documentation necessary to build your own honeypot system and contribute to our [Sicherheitstacho](https://sicherheitstacho.eu).
 
-The source code and configuration files are stored in individual GitHub repositories, which are linked below. The docker images are pre-configured for the T-Pot environment. If you want to run the docker images separately, make sure you study the docker-compose configuration (`/opt/tpot/etc/tpot.yml`) and the T-Pot systemd script (`/etc/systemd/system/tpot.service`), as they provide a good starting point for implementing changes.
+The source code and configuration files are fully stored in the T-Pot GitHub repository. The docker images are pre-configured for the T-Pot environment. If you want to run the docker images separately, make sure you study the docker-compose configuration (`/opt/tpot/etc/tpot.yml`) and the T-Pot systemd script (`/etc/systemd/system/tpot.service`), as they provide a good starting point for implementing changes.
 
-The individual docker configurations are located in the following GitHub repositories:
-
-- [conpot](https://github.com/dtag-dev-sec/conpot)
-- [cowrie](https://github.com/dtag-dev-sec/cowrie)
-- [dionaea](https://github.com/dtag-dev-sec/dionaea)
-- [elasticpot](https://github.com/dtag-dev-sec/elasticpot)
-- [elk-stack](https://github.com/dtag-dev-sec/elk)
-- [emobility](https://github.com/dtag-dev-sec/emobility)
-- [ewsposter](https://github.com/dtag-dev-sec/ews)
-- [glastopf](https://github.com/dtag-dev-sec/glastopf)
-- [honeytrap](https://github.com/dtag-dev-sec/honeytrap)
-- [mailoney](https://github.com/dtag-dev-sec/mailoney)
-- [netdata](https://github.com/dtag-dev-sec/netdata)
-- [portainer](https://github.com/dtag-dev-sec/ui-for-docker)
-- [rdpy](https://github.com/dtag-dev-sec/rdpy)
-- [spiderfoot](https://github.com/dtag-dev-sec/spiderfoot)
-- [suricata & p0f](https://github.com/dtag-dev-sec/suricata)
-- [vnclowpot](https://github.com/dtag-dev-sec/vnclowpot)
+The individual docker configurations are located in the [docker folder](https://github.com/dtag-dev-sec/tpotce/tree/master/docker).
 
 <a name="requirements"></a>
 # System Requirements
 Depending on your installation type, whether you install on [real hardware](#hardware) or in a [virtual machine](#vm), make sure your designated T-Pot system meets the following requirements:
 
-##### T-Pot Installation (Cowrie, Dionaea, ElasticPot, Glastopf, Honeytrap, Mailoney, Rdpy, Vnclowpot, ELK, Suricata+P0f & Tools)
-When installing the T-Pot ISO image, make sure the target system (physical/virtual) meets the following minimum requirements:
+##### T-Pot Standard Installation
+- Honeypots: ciscoasa, conpot, cowrie, dionaea, elasticpot, heralding, honeytrap, mailoney, rdpy, snare, tanner and vnclowpot
+- Tools: cockpit, cyberchef, ELK, elasticsearch head, ewsposter, NGINX, spiderfoot, p0f and suricata
 
-- 4 GB RAM (6-8 GB recommended)
-- 64 GB SSD (128 GB SSD recommended)
+- 6-8 GB RAM (less RAM is possible but might introduce swapping)
+- 128 GB SSD (smaller is possible but limits the capacity of storing events)
 - Network via DHCP
 - A working, non-proxied, internet connection
 
-##### Honeypot Installation (Cowrie, Dionaea, ElasticPot, Glastopf, Honeytrap, Mailoney, Rdpy, Vnclowpot)
-When installing the T-Pot ISO image, make sure the target system (physical/virtual) meets the following minimum requirements:
+##### Sensor Installation
+- Honeypots: ciscoasa, conpot, cowrie, dionaea, elasticpot, heralding, honeytrap, mailoney, rdpy, snare, tanner and vnclowpot
+- Tools: cockpit
 
-- 3 GB RAM (4-6 GB recommended)
-- 64 GB SSD (64 GB SSD recommended)
+- 6-8 GB RAM (less RAM is possible but might introduce swapping)
+- 128 GB SSD (smaller is possible but limits the capacity of storing events)
 - Network via DHCP
 - A working, non-proxied, internet connection
 
-##### Industrial Installation (ConPot, eMobility, ELK, Suricata+P0f & Tools)
-When installing the T-Pot ISO image, make sure the target system (physical/virtual) meets the following minimum requirements:
+##### Industrial Installation
+- Honeypots: conpot, rdpy, vnclowpot
+- Tools: cockpit, cyberchef, ELK, elasticsearch head, ewsposter, NGINX, spiderfoot, p0f and suricata
 
-- 4 GB RAM (8 GB recommended)
-- 64 GB SSD (128 GB SSD recommended)
+- 6-8 GB RAM (less RAM is possible but might introduce swapping)
+- 128 GB SSD (smaller is possible but limits the capacity of storing events)
 - Network via DHCP
 - A working, non-proxied, internet connection
 
-##### Everything Installation (Everything, all of the above)
-When installing the T-Pot ISO image, make sure the target system (physical/virtual) meets the following minimum requirements:
+##### Collector Installation (because sometimes all you want to do is catching credentials)
+- Honeypots: heralding
+- Tools: cockpit, cyberchef, ELK, elasticsearch head, ewsposter, NGINX, spiderfoot, p0f and suricata
 
-- 8+ GB RAM
-- 128+ GB SSD
+- 6-8 GB RAM (less RAM is possible but might introduce swapping)
+- 128 GB SSD (smaller is possible but limits the capacity of storing events)
+- Network via DHCP
+- A working, non-proxied, internet connection
+
+##### NextGen Installation (Glutton instead of Honeytrap)
+- Honeypots: ciscoasa, conpot, cowrie, dionaea, elasticpot, glutton, heralding, mailoney, rdpy, snare, tanner and vnclowpot
+- Tools: cockpit, cyberchef, ELK, elasticsearch head, ewsposter, NGINX, spiderfoot, p0f and suricata
+
+- 6-8 GB RAM (less RAM is possible but might introduce swapping)
+- 128 GB SSD (smaller is possible but limits the capacity of storing events)
+- Network via DHCP
+- A working, non-proxied, internet connection
+
+##### Legacy Installation (honeypots based on Standard Installation of T-Pot 17.10)
+- Honeypots: cowrie, dionaea, elasticpot, glastopf, honeytrap, mailoney, rdpy and vnclowpot
+- Tools: cockpit, cyberchef, ELK, elasticsearch head, ewsposter, NGINX, spiderfoot, p0f and suricata
+
+- 6-8 GB RAM (less RAM is possible but might introduce swapping)
+- 128 GB SSD (smaller is possible but limits the capacity of storing events)
 - Network via DHCP
 - A working, non-proxied, internet connection
 
@@ -226,7 +224,7 @@ When installing the T-Pot ISO image, make sure the target system (physical/virtu
 # Installation
 The installation of T-Pot is straight forward and heavily depends on a working, transparent and non-proxied up and running internet connection. Otherwise the installation **will fail!**
 
-Firstly, decide if you want to download our prebuilt installation ISO image from [GitHub](https://github.com/dtag-dev-sec/tpotce/releases) ***or*** [create it yourself](#createiso).
+Firstly, decide if you want to download our prebuilt installation ISO image from [GitHub](https://github.com/dtag-dev-sec/tpotce/releases), [create it yourself](#createiso) ***or*** [post-install on a existing Ubuntu Server 18.04 LTS](#postinstall).
 
 Secondly, decide where you want to let the system run: [real hardware](#hardware) or in a [virtual machine](#vm)?
 
@@ -240,7 +238,7 @@ You can download the prebuilt installation image from [GitHub](https://github.co
 For transparency reasons and to give you the ability to customize your install, we provide you the [ISO Creator](https://github.com/dtag-dev-sec/tpotce) that enables you to create your own ISO installation image.
 
 **Requirements to create the ISO image:**
-- Ubuntu 16.04 LTS or newer as host system (others *may* work, but remain untested)
+- Ubuntu 18.04 LTS or newer as host system (others *may* work, but *remain* untested)
 - 4GB of free memory  
 - 32GB of free storage
 - A working internet connection
@@ -265,11 +263,11 @@ You may want to run T-Pot in a virtualized environment. The virtual system confi
 
 We successfully tested T-Pot with [VirtualBox](https://www.virtualbox.org) and [VMWare](http://www.vmware.com) with just little modifications to the default machine configurations.
 
-It is important to make sure you meet the [system requirements](#requirements) and assign a virtual harddisk >=64 GB, >=4 GB RAM and bridged networking to T-Pot.
+It is important to make sure you meet the [system requirements](#requirements) and assign a virtual harddisk and RAM according to the requirements while making sure networking is bridged.
 
 You need to enable promiscuous mode for the network interface for suricata and p0f to work properly. Make sure you enable it during configuration.
 
-If you want to use a wifi card as primary NIC for T-Pot, please be aware of the fact that not all network interface drivers support all wireless cards. E.g. in VirtualBox, you then have to choose the *"MT SERVER"* model of the NIC.
+If you want to use a wifi card as a primary NIC for T-Pot, please be aware of the fact that not all network interface drivers support all wireless cards. E.g. in VirtualBox, you then have to choose the *"MT SERVER"* model of the NIC.
 
 Lastly, mount the `tpot.iso` ISO to the VM and continue with the installation.<br>
 
@@ -283,82 +281,143 @@ If you decide to run T-Pot on dedicated hardware, just follow these steps:
 Whereas most CD burning tools allow you to burn from ISO images, the procedure to create a bootable USB stick from an ISO image depends on your system. There are various Windows GUI tools available, e.g. [this tip](http://www.ubuntu.com/download/desktop/create-a-usb-stick-on-windows) might help you.<br> On [Linux](http://askubuntu.com/questions/59551/how-to-burn-a-iso-to-a-usb-device) or [MacOS](http://www.ubuntu.com/download/desktop/create-a-usb-stick-on-mac-osx) you can use the tool *dd* or create the USB stick with T-Pot's [ISO Creator](https://github.com/dtag-dev-sec).
 2. Boot from the USB stick and install.
 
-*Please note*: We will ensure the compatibility with the Intel NUC platform, as we really like the form factor, looks and build quality.
+*Please note*: We will ensure the compatibility with the Intel NUC platform, as we really like the form factor, looks and build quality. Other platforms **remain untested**.
+
+<a name="postinstall"></a>
+## Post-Install Manual
+In some cases it is necessary to install Ubuntu Server 18.04 LTS on your own:
+ - Cloud provider does not offer mounting ISO images.
+ - Hardware setup needs special drivers and / or kernels.
+ - Within your company you have to setup special policies, software etc.
+ - You just like to stay on top of things.
+
+While the T-Pot-Autoinstaller served us perfectly well in the past we decided to include the feature directly into T-Pot and its Universal Installer.
+
+Just follow these steps:
+
+```
+git clone https://github.com/dtag-dev-sec/tpotce
+cd tpotce/iso/installer/
+./install.sh --type=user
+```
+
+The installer will now start and guide you through the install process.
+
+<a name="postinstallauto"></a>
+## Post-Install Auto
+You can also let the installer run automatically if you provide your own `tpot.conf`. A example is available in `tpotce/iso/installer/tpot.conf.dist`. This should make things easier in case you want to automate the installation i.e. with **Ansible**.
+
+Just follow these steps while adjusting `tpot.conf` to your needs:
+
+```
+git clone https://github.com/dtag-dev-sec/tpotce
+cd tpotce/iso/installer/
+cp tpot.conf.dist tpot.conf
+./install.sh --type=auto --conf=tpot.conf
+```
+
+The installer will start automatically and guide you through the install process.
 
 <a name="firstrun"></a>
 ## First Run
-The installation requires very little interaction, only a locale and keyboard setting has to be answered for the basic linux installation. The system will reboot and please maintain an active internet connection. The T-Pot installer will start and ask you for an installation type, password for the **tsec** user and credentials for a **web user**. Everything else will be configured automatically. All docker images and other componenents will be downloaded. Depending on your network connection and the chosen installation type, the installation may take some time. During our tests (50Mbit down, 10Mbit up), the installation is usually finished within a 30 minute timeframe.
+The installation requires very little interaction, only a locale and keyboard setting have to be answered for the basic linux installation. The system will reboot and please maintain the active internet connection. The T-Pot installer will start and ask you for an installation type, password for the **tsec** user and credentials for a **web user**. Everything else will be configured automatically. All docker images and other componenents will be downloaded. Depending on your network connection and the chosen installation type, the installation may take some time. During our tests (250Mbit down, 40Mbit up), the installation was usually finished within a 15-30 minute timeframe.
 
-Once the installation is finished, the system will automatically reboot and you will be presented with the T-Pot login screen. On the console you may login with the **tsec** user:
+Once the installation is finished, the system will automatically reboot and you will be presented with the T-Pot login screen. On the console you may login with:
 
-- user: **tsec**
-- pass: **password you chose during the installation**
+- user: **[tsec or user]** *you chose during one of the post install methods*
+- pass: **[password]** *you chose during the installation*
 
 All honeypot services are preconfigured and are starting automatically.
 
-You can also login from your browser: ``https://<your.ip>:64297``
+You can login from your browser and access the Admin UI: `https://<your.ip>:64294` or via SSH to access the command line: `ssh -l tsec -p 64295 <your.ip>`
 
-- user: **user you chose during the installation**
-- pass: **password you chose during the installation**
+- user: **[tsec or user]** *you chose during one of the post install methods*
+- pass: **[password]** *you chose during the installation*
+
+You can also login from your browser and access the Web UI: `https://<your.ip>:64297`
+- user: **[user]** *you chose during the installation*
+- pass: **[password]** *you chose during the installation*
 
 
 <a name="placement"></a>
 # System Placement
-Make sure your system is reachable through the internet. Otherwise it will not capture any attacks, other than the ones from your  internal network! We recommend you put it in an unfiltered zone, where all TCP and UDP traffic is forwarded to T-Pot's network interface.
+Make sure your system is reachable through the internet. Otherwise it will not capture any attacks, other than the ones from your  internal network! We recommend you put it in an unfiltered zone, where all TCP and UDP traffic is forwarded to T-Pot's network interface. However to avoid fingerprinting you can put T-Pot behind a firewall and forward all TCP / UDP traffic in the port range of 1-64000 to T-Pot while allowing access to ports > 64000 only from trusted IPs.
 
 A list of all relevant ports is available as part of the [Technical Concept](#concept)
 <br>
 
 Basically, you can forward as many TCP ports as you want, as honeytrap dynamically binds any TCP port that is not covered by the other honeypot daemons.
 
+In case you need external Admin UI access, forward TCP port 64294 to T-Pot, see below.
 In case you need external SSH access, forward TCP port 64295 to T-Pot, see below.
-In case you need external web access, forward TCP port 64297 to T-Pot, see below.
+In case you need external Web UI access, forward TCP port 64297 to T-Pot, see below.
 
 T-Pot requires outgoing git, http, https connections for updates (Ubuntu, Docker, GitHub, PyPi) and attack submission (ewsposter, hpfeeds). Ports and availability may vary based on your geographical location.
 
+<a name="updates"></a>
+# Updates
+For the ones of you who want to live on the bleeding edge of T-Pot development we introduced an update feature which will allow you to update all T-Pot relevant files to be up to date with the T-Pot master branch.
+**If you made any relevant changes to the T-Pot relevant config files make sure to create a backup first.**
+- The Update script will
+ - **merciless** overwrite local changes to be in sync with the T-Pot master branch
+ - upgrade the system to the latest kernel within Ubuntu 18.04.x LTS
+ - upgrade the system to the latest packages available within Ubuntu 18.04.x LTS
+ - update all resources to be en par with the T-Pot master branch
+ - ensure all T-Pot relevant system files will be patched / copied into original T-Pot state
+
+You simply run the update script:
+```
+cd /opt/tpot/
+./update.sh -y
+```
+
+**Despite all our efforts please be reminded that updates sometimes may have unforeseen consequences. Please create a backup of the machine or the files with the most value to your work.**  
+
 <a name="options"></a>
 # Options
-The system is designed to run without any interaction or maintenance and automatically contribute to the community.<br>
+The system is designed to run without any interaction or maintenance and automatically contributes to the community.<br>
 We know, for some this may not be enough. So here come some ways to further inspect the system and change configuration parameters.
 
 <a name="ssh"></a>
 ## SSH and web access
-By default, the SSH daemon only allows access on **tcp/64295** with a user / password combination from RFC1918 networks. However, if you want to be able to login remotely via SSH you need to put your SSH keys on the host as described below.<br>
-It is configured to prevent password login from official IP addresses and pubkey-authentication must be used. Copy your SSH keyfile to `/home/tsec/.ssh/authorized_keys` and set the appropriate permissions (`chmod 600 authorized_keys`) as well as the correct ownership (`chown tsec:tsec authorized_keys`).
+By default, the SSH daemon allows access on **tcp/64295** with a user / password combination and prevents credential brute forcing attempts using `fail2ban`. This also counts for Admin UI (**tcp/64294**) and Web UI (**tcp/64297**) access.<br>
 
-If you do not have a SSH client at hand and still want to access the machine via SSH you can do so by directing your browser to `https://<your.ip>:64297`, enter
+If you do not have a SSH client at hand and still want to access the machine via command line you can do so by accessing the Admin UI from `https://<your.ip>:64294`, enter
 
-- user: **user you chose during the installation**
-- pass: **password you chose during the installation**
+- user: **[tsec or user]** *you chose during one of the post install methods*
+- pass: **[password]** *you chose during the installation*
 
-and choose **WebTTY** from the navigation bar. You will be prompted to allow access for this connection and enter the password for the user **tsec**.
-
-![WebTTY](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/doc/webssh.png)
+![Cockpit Terminal](doc/cockpit3.png)
 
 <a name="kibana"></a>
 ## Kibana Dashboard
-Just open a web browser and access and connect to `https://<your.ip>:64297`, enter
+Just open a web browser and connect to `https://<your.ip>:64297`, enter
 
-- user: **user you chose during the installation**
-- pass: **password you chose during the installation**
+- user: **[user]** *you chose during the installation*
+- pass: **[password]** *you chose during the installation*
 
 and **Kibana** will automagically load. The Kibana dashboard can be customized to fit your needs. By default, we haven't added any filtering, because the filters depend on your setup. E.g. you might want to filter out your incoming administrative ssh connections and connections to update servers.
 
-![Dashbaord](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/doc/dashboard.png)
+![Dashbaord](doc/kibana.png)
 
 <a name="tools"></a>
 ## Tools
 We included some web based management tools to improve and ease up on your daily tasks.
 
-![ES Head Plugin](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/doc/headplugin.png)
-![Netdata](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/doc/netdata.png)
-![Portainer](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/doc/dockerui.png)
-![Spiderfoot](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/doc/spiderfoot.png)
+![Cockpit Overview](doc/cockpit1.png)
+
+![Cockpit Containers](doc/cockpit2.png)
+
+![Cyberchef](doc/cyberchef.png)
+
+![ES Head Plugin](doc/headplugin.png)
+
+![Spiderfoot](doc/spiderfoot.png)
 
 
 <a name="maintenance"></a>
 ## Maintenance
-As mentioned before, the system was designed to be low maintenance. Basically, there is nothing you have to do but let it run.
+As mentioned before, the system is designed to be low maintenance. Basically, there is nothing you have to do but let it run.
 
 If you run into any problems, a reboot may fix it :bowtie:
 
@@ -366,8 +425,8 @@ If new versions of the components involved appear, we will test them and build n
 
 <a name="submission"></a>
 ## Community Data Submission
-We provide T-Pot in order to make it accessible to all parties interested in honeypot deployment. By default, the data captured is submitted to a community backend. This community backend uses the data to feed a [community data view](http://sicherheitstacho.eu/?peers=communityPeers), a separate channel on our own [Sicherheitstacho](http://sicherheitstacho.eu), which is powered by our own set of honeypots.
-You may opt out the submission to our community server by removing the `# Ewsposter service` from `/opt/tpot/etc/tpot.yml`:
+We provide T-Pot in order to make it accessible to all parties interested in honeypot deployment. By default, the captured data is submitted to a community backend. This community backend uses the data to feed [Sicherheitstacho](https://sicherheitstacho.eu.
+You may opt out of the submission by removing the `# Ewsposter service` from `/opt/tpot/etc/tpot.yml`:
 1. Stop T-Pot services: `systemctl stop tpot`
 2. Remove Ewsposter service: `vi /opt/tpot/etc/tpot.yml`
 3. Remove the following lines, save and exit vi (`:x!`):<br>
@@ -378,7 +437,7 @@ You may opt out the submission to our community server by removing the `# Ewspos
     restart: always
     networks:
      - ewsposter_local
-    image: "dtagdevsec/ewsposter:1710"
+    image: "dtagdevsec/ewsposter:1810"
     volumes:
      - /data:/data
      - /data/ews/conf/ews.ip:/opt/ewsposter/ews.ip
@@ -393,9 +452,6 @@ We encourage you not to disable the data submission as it is the main purpose of
 # Roadmap
 As with every development there is always room for improvements ...
 
-- Introduce new honeypots
-- Improve automatic updates
-
 Some features may be provided with updated docker images, others may require some hands on from your side.
 
 You are always invited to participate in development on our [GitHub](https://github.com/dtag-dev-sec/tpotce) page.
@@ -405,7 +461,7 @@ You are always invited to participate in development on our [GitHub](https://git
 - We don't have access to your system. So we cannot remote-assist when you break your configuration. But you can simply reinstall.
 - The software was designed with best effort security, not to be in stealth mode. Because then, we probably would not be able to provide those kind of honeypot services.
 - You install and you run within your responsibility. Choose your deployment wisely as a system compromise can never be ruled out.
-- Honeypots should - by design - not host any sensitive data. Make sure you don't add any.
+- Honeypots should - by design - may not host any sensitive data. Make sure you don't add any.
 - By default, your data is submitted to the community dashboard. You can disable this in the config. But hey, wouldn't it be better to contribute to the community?
 
 <a name="faq"></a>
@@ -417,20 +473,14 @@ Please report any issues or questions on our [GitHub issue list](https://github.
 We provide the software **as is** in a Community Edition format. T-Pot is designed to run out of the box and with zero maintenance involved. <br>
 We hope you understand that we cannot provide support on an individual basis. We will try to address questions, bugs and problems on our [GitHub issue list](https://github.com/dtag-dev-sec/tpotce/issues).
 
-For general feedback you can write to cert @ telekom.de.
-
 <a name="licenses"></a>
 # Licenses
 The software that T-Pot is built on uses the following licenses.
-<br>GPLv2: [conpot (by Lukas Rist)](https://github.com/mushorg/conpot/blob/master/LICENSE.txt), [dionaea](https://github.com/DinoTools/dionaea/blob/master/LICENSE), [honeytrap (by Tillmann Werner)](https://github.com/armedpot/honeytrap/blob/master/LICENSE), [suricata](http://suricata-ids.org/about/open-source/)
-<br>GPLv3: [elasticpot (by Markus Schmall)](https://github.com/schmalle/ElasticPot), [emobility (by Mohamad Sbeiti)](https://github.com/dtag-dev-sec/emobility/blob/master/LICENSE), [ewsposter (by Markus Schroer)](https://github.com/dtag-dev-sec/ews/), [glastopf (by Lukas Rist)](https://github.com/glastopf/glastopf/blob/master/GPL), [rdpy](https://github.com/citronneur/rdpy/blob/master/LICENSE), [netdata](https://github.com/firehol/netdata/blob/master/LICENSE.md)
-<br>Apache 2 License: [elasticsearch](https://github.com/elasticsearch/elasticsearch/blob/master/LICENSE.txt), [logstash](https://github.com/elasticsearch/logstash/blob/master/LICENSE), [kibana](https://github.com/elasticsearch/kibana/blob/master/LICENSE.md), [docker](https://github.com/docker/docker/blob/master/LICENSE), [elasticsearch-head](https://github.com/mobz/elasticsearch-head/blob/master/LICENCE)
-<br>MIT License: [ctop](https://github.com/bcicen/ctop/blob/master/LICENSE), [wetty](https://github.com/krishnasrinivas/wetty/blob/master/LICENSE)
-<br>zlib License: [vnclowpot](https://github.com/magisterquis/vnclowpot/blob/master/LICENSE)
-<br>[cowrie (copyright disclaimer by Upi Tamminen)](https://github.com/micheloosterhof/cowrie/blob/master/doc/COPYRIGHT)
-<br>[mailoney](https://github.com/awhitehatter/mailoney)
-<br>[Ubuntu licensing](http://www.ubuntu.com/about/about-ubuntu/licensing)
-<br>[Portainer](https://github.com/portainer/portainer/blob/develop/LICENSE)
+<br>GPLv2: [conpot)](https://github.com/mushorg/conpot/blob/master/LICENSE.txt), [dionaea](https://github.com/DinoTools/dionaea/blob/master/LICENSE), [honeytrap](https://github.com/armedpot/honeytrap/blob/master/LICENSE), [suricata](http://suricata-ids.org/about/open-source/)
+<br>GPLv3: [elasticpot](https://github.com/schmalle/ElasticPot), [ewsposter](https://github.com/dtag-dev-sec/ews/), [glastopf](https://github.com/glastopf/glastopf/blob/master/GPL), [rdpy](https://github.com/citronneur/rdpy/blob/master/LICENSE), [heralding](https://github.com/johnnykv/heralding/blob/master/LICENSE.txt), [snare](https://github.com/mushorg/snare/blob/master/LICENSE), [tanner](https://github.com/mushorg/snare/blob/master/LICENSE)
+<br>Apache 2 License: [cyberchef](https://github.com/gchq/CyberChef/blob/master/LICENSE), [elasticsearch](https://github.com/elasticsearch/elasticsearch/blob/master/LICENSE.txt), [logstash](https://github.com/elasticsearch/logstash/blob/master/LICENSE), [kibana](https://github.com/elasticsearch/kibana/blob/master/LICENSE.md), [docker](https://github.com/docker/docker/blob/master/LICENSE), [elasticsearch-head](https://github.com/mobz/elasticsearch-head/blob/master/LICENCE)
+<br>MIT license: [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot/blob/master/LICENSE), [ctop](https://github.com/bcicen/ctop/blob/master/LICENSE), [glutton](https://github.com/mushorg/glutton/blob/master/LICENSE)
+<br> Other: [cowrie](https://github.com/micheloosterhof/cowrie/blob/master/LICENSE.md), [mailoney](https://github.com/awhitehatter/mailoney), [Ubuntu licensing](http://www.ubuntu.com/about/about-ubuntu/licensing)
 
 <a name="credits"></a>
 # Credits
@@ -438,6 +488,8 @@ Without open source and the fruitful development community we are proud to be a
 
 ### The developers and development communities of
 
+* [ciscoasa](https://github.com/Cymmetria/ciscoasa_honeypot/graphs/contributors)
+* [cockpit](https://github.com/cockpit-project/cockpit/graphs/contributors)
 * [conpot](https://github.com/mushorg/conpot/graphs/contributors)
 * [cowrie](https://github.com/micheloosterhof/cowrie/graphs/contributors)
 * [dionaea](https://github.com/DinoTools/dionaea/graphs/contributors)
@@ -445,22 +497,22 @@ Without open source and the fruitful development community we are proud to be a
 * [elasticpot](https://github.com/schmalle/ElasticPot/graphs/contributors)
 * [elasticsearch](https://github.com/elastic/elasticsearch/graphs/contributors)
 * [elasticsearch-head](https://github.com/mobz/elasticsearch-head/graphs/contributors)
-* [emobility](https://github.com/dtag-dev-sec/emobility/graphs/contributors)
 * [ewsposter](https://github.com/armedpot/ewsposter/graphs/contributors)
 * [glastopf](https://github.com/mushorg/glastopf/graphs/contributors)
+* [glutton](https://github.com/mushorg/glutton/graphs/contributors)
+* [heralding](https://github.com/johnnykv/heralding/graphs/contributors)
 * [honeytrap](https://github.com/armedpot/honeytrap/graphs/contributors)
 * [kibana](https://github.com/elastic/kibana/graphs/contributors)
 * [logstash](https://github.com/elastic/logstash/graphs/contributors)
 * [mailoney](https://github.com/awhitehatter/mailoney)
-* [netdata](https://github.com/firehol/netdata/graphs/contributors)
+* [medpot](https://github.com/schmalle/medpot/graphs/contributors)
 * [p0f](http://lcamtuf.coredump.cx/p0f3/)
-* [portainer](https://github.com/portainer/portainer/graphs/contributors)
 * [rdpy](https://github.com/citronneur/rdpy)
 * [spiderfoot](https://github.com/smicallef/spiderfoot)
+* [snare](https://github.com/mushorg/snare/graphs/contributors)
+* [tanner](https://github.com/mushorg/tanner/graphs/contributors)
 * [suricata](https://github.com/inliniac/suricata/graphs/contributors)
 * [ubuntu](http://www.ubuntu.com/)
-* [vnclowpot](https://github.com/magisterquis/vnclowpot)
-* [wetty](https://github.com/krishnasrinivas/wetty/graphs/contributors)
 
 ### The following companies and organizations
 * [canonical](http://www.canonical.com/)
@@ -478,4 +530,4 @@ We will be releasing a new version of T-Pot about every 6-12 months.
 <a name="funfact"></a>
 # Fun Fact
 
-Coffee just does not cut it anymore which is why we needed a different caffeine source and consumed *242* bottles of [Club Mate](https://de.wikipedia.org/wiki/Club-Mate) during the development of T-Pot 17.10 😇
+In an effort of saving the environment we are now brewing our own Mate Ice Tea and consumed 235 liters so far for the T-Pot 18.11 development 😇

bin/backup_es_folders.sh

@@ -5,7 +5,7 @@ myES="http://127.0.0.1:64298/"
 myESSTATUS=$(curl -s -XGET ''$myES'_cluster/health' | jq '.' | grep -c green)
 if ! [ "$myESSTATUS" = "1" ]
   then
-    echo "### Elasticsearch is not available, try starting via 'systemctl start elk'."
+    echo "### Elasticsearch is not available, try starting via 'systemctl start tpot'."
     exit
   else
     echo "### Elasticsearch is available, now continuing."
@@ -16,7 +16,7 @@ fi
 myCOUNT=1
 myDATE=$(date +%Y%m%d%H%M)
 myELKPATH="/data/elk/data"
-myKIBANAINDEXNAME=$(curl -s -XGET ''$myES'_cat/indices/' | grep .kibana | awk '{ print $4 }')
+myKIBANAINDEXNAME=$(curl -s -XGET ''$myES'_cat/indices/' | grep -w ".kibana_1" | awk '{ print $4 }')
 myKIBANAINDEXPATH=$myELKPATH/nodes/0/indices/$myKIBANAINDEXNAME
 
 # Let's ensure normal operation on exit or if interrupted ...

bin/clean.sh

@@ -32,10 +32,14 @@ fuLOGROTATE () {
   local myHONEYTRAPATTACKSTGZ="/data/honeytrap/attacks.tgz"
   local myHONEYTRAPDL="/data/honeytrap/downloads/"
   local myHONEYTRAPDLTGZ="/data/honeytrap/downloads.tgz"
+  local myTANNERF="/data/tanner/files/"
+  local myTANNERFTGZ="/data/tanner/files.tgz"
 
 # Ensure correct permissions and ownerships for logrotate to run without issues
 chmod 760 /data/ -R
 chown tpot:tpot /data -R
+chmod 644 /data/nginx/conf -R
+chmod 644 /data/nginx/cert -R
 
 # Run logrotate with force (-f) first, so the status file can be written and race conditions (with tar) be avoided
 logrotate -f -s $mySTATUS $myCONF
@@ -47,23 +51,32 @@ if [ "$(fuEMPTY $myDIONAEABI)" != "0" ]; then tar cvfz $myDIONAEABITGZ $myDIONAE
 if [ "$(fuEMPTY $myDIONAEABIN)" != "0" ]; then tar cvfz $myDIONAEABINTGZ $myDIONAEABIN; fi
 if [ "$(fuEMPTY $myHONEYTRAPATTACKS)" != "0" ]; then tar cvfz $myHONEYTRAPATTACKSTGZ $myHONEYTRAPATTACKS; fi
 if [ "$(fuEMPTY $myHONEYTRAPDL)" != "0" ]; then tar cvfz $myHONEYTRAPDLTGZ $myHONEYTRAPDL; fi
+if [ "$(fuEMPTY $myTANNERF)" != "0" ]; then tar cvfz $myTANNERFTGZ $myTANNERF; fi
 
 # Ensure correct permissions and ownership for previously created archives
-chmod 760 $myCOWRIETTYTGZ $myCOWRIEDLTGZ $myDIONAEABITGZ $myDIONAEABINTGZ $myHONEYTRAPATTACKSTGZ $myHONEYTRAPDLTGZ
-chown tpot:tpot $myCOWRIETTYTGZ $myCOWRIEDLTGZ $myDIONAEABITGZ $myDIONAEABINTGZ $myHONEYTRAPATTACKSTGZ $myHONEYTRAPDLTGZ
+chmod 760 $myCOWRIETTYTGZ $myCOWRIEDLTGZ $myDIONAEABITGZ $myDIONAEABINTGZ $myHONEYTRAPATTACKSTGZ $myHONEYTRAPDLTGZ $myTANNERFTGZ
+chown tpot:tpot $myCOWRIETTYTGZ $myCOWRIEDLTGZ $myDIONAEABITGZ $myDIONAEABINTGZ $myHONEYTRAPATTACKSTGZ $myHONEYTRAPDLTGZ $myTANNERFTGZ
 
 # Need to remove subfolders since too many files cause rm to exit with errors
-rm -rf $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL
+rm -rf $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL $myTANNERF
 
 # Recreate subfolders with correct permissions and ownership
-mkdir -p $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL
-chmod 760 $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL 
-chown tpot:tpot $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL
+mkdir -p $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL $myTANNERF
+chmod 760 $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL $myTANNERF
+chown tpot:tpot $myCOWRIETTYLOGS $myCOWRIEDL $myDIONAEABI $myDIONAEABIN $myHONEYTRAPATTACKS $myHONEYTRAPDL $myTANNERF
 
 # Run logrotate again to account for previously created archives - DO NOT FORCE HERE!
 logrotate -s $mySTATUS $myCONF
 }
 
+# Let's create a function to clean up and prepare ciscoasa data
+fuCISCOASA () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/ciscoasa/*; fi
+  mkdir -p /data/ciscoasa/log
+  chmod 760 /data/ciscoasa -R
+  chown tpot:tpot /data/ciscoasa -R
+}
+
 # Let's create a function to clean up and prepare conpot data
 fuCONPOT () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/conpot/*; fi
@@ -101,27 +114,35 @@ fuELK () {
   # ELK data will be kept for <= 90 days, check /etc/crontab for curator modification
   # ELK daemon log files will be removed
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/elk/log/*; fi
-  mkdir -p /data/elk 
+  mkdir -p /data/elk
   chmod 760 /data/elk -R
   chown tpot:tpot /data/elk -R
 }
 
-# Let's create a function to clean up and prepare emobility data
-fuEMOBILITY () {
-  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/emobility/*; fi
-  mkdir -p /data/emobility/log
-  chmod 760 /data/emobility -R
-  chown tpot:tpot /data/emobility -R
-}
-
 # Let's create a function to clean up and prepare glastopf data
 fuGLASTOPF () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/glastopf/*; fi
-  mkdir -p /data/glastopf
+  mkdir -p /data/glastopf/db /data/glastopf/log
   chmod 760 /data/glastopf -R
   chown tpot:tpot /data/glastopf -R
 }
 
+# Let's create a function to clean up and prepare glastopf data
+fuGLUTTON () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/glutton/*; fi
+  mkdir -p /data/glutton/log
+  chmod 760 /data/glutton -R
+  chown tpot:tpot /data/glutton -R
+}
+
+# Let's create a function to clean up and prepare heralding data
+fuHERALDING () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/heralding/*; fi
+  mkdir -p /data/heralding/log
+  chmod 760 /data/heralding -R
+  chown tpot:tpot /data/heralding -R
+}
+
 # Let's create a function to clean up and prepare honeytrap data
 fuHONEYTRAP () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/honeytrap/*; fi
@@ -138,6 +159,22 @@ fuMAILONEY () {
   chown tpot:tpot /data/mailoney/ -R
 }
 
+# Let's create a function to clean up and prepare mailoney data
+fuMEDPOT () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/medpot/*; fi
+  mkdir -p /data/medpot/log/
+  chmod 760 /data/medpot/ -R
+  chown tpot:tpot /data/medpot/ -R
+}
+
+# Let's create a function to clean up nginx logs
+fuNGINX () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/nginx/log/*; fi
+  touch /data/nginx/log/error.log
+  chmod 644 /data/nginx/conf -R
+  chmod 644 /data/nginx/cert -R
+}
+
 # Let's create a function to clean up and prepare rdpy data
 fuRDPY () {
   if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/rdpy/*; fi
@@ -170,15 +207,14 @@ fuP0F () {
   chown tpot:tpot -R /data/p0f
 }
 
-# Let's create a function to clean up and prepare vnclowpot data
-fuVNCLOWPOT () {
-  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/vnclowpot/*; fi
-  mkdir -p /data/vnclowpot/log/
-  chmod 760 /data/vnclowpot/ -R
-  chown tpot:tpot /data/vnclowpot/ -R
+# Let's create a function to clean up and prepare p0f data
+fuTANNER () {
+  if [ "$myPERSISTENCE" != "on" ]; then rm -rf /data/tanner/*; fi
+  mkdir -p /data/tanner/log /data/tanner/files
+  chmod 760 -R /data/tanner
+  chown tpot:tpot -R /data/tanner
 }
 
-
 # Avoid unwanted cleaning
 if [ "$myPERSISTENCE" = "" ];
   then
@@ -201,19 +237,22 @@ if [ "$myPERSISTENCE" = "on" ];
     fuLOGROTATE
   else
     echo "Cleaning up and preparing data folders."
+    fuCISCOASA
     fuCONPOT
     fuCOWRIE
     fuDIONAEA
     fuELASTICPOT
     fuELK
-    fuEMOBILITY
     fuGLASTOPF
+    fuGLUTTON
+    fuHERALDING
     fuHONEYTRAP
     fuMAILONEY
+    fuMEDPOT
+    fuNGINX
     fuRDPY
     fuSPIDERFOOT
     fuSURICATA
     fuP0F
-    fuVNCLOWPOT
+    fuTANNER
   fi
-

bin/dps.sh

@@ -1,44 +1,15 @@
 #/bin/bash
-# Show current status of all running containers
+# Show current status of T-Pot containers
 myPARAM="$1"
-myIMAGES="$(cat /opt/tpot/etc/tpot.yml | grep -v '#' | grep container_name | cut -d: -f2)"
+myCONTAINERS="$(cat /opt/tpot/etc/tpot.yml | grep -v '#' | grep container_name | cut -d: -f2 | sort | tr -d " ")"
 myRED=""
 myGREEN=""
 myBLUE=""
 myWHITE=""
 myMAGENTA=""
 
-function fuCONTAINERSTATUS {
-local myNAME="$1"
-local mySTATUS="$(/usr/bin/docker ps -f name=$myNAME --format "table {{.Status}}" -f status=running -f status=exited | tail -n 1)"
-myDOWN="$(echo "$mySTATUS" | grep -o -E "(STATUS|NAMES|Exited)")"
-
-case "$myDOWN" in
-  STATUS)
-    mySTATUS="$myRED"DOWN"$myWHITE"
-  ;;
-  NAMES)
-    mySTATUS="$myRED"DOWN"$myWHITE"
-  ;;
-  Exited)
-    mySTATUS="$myRED$mySTATUS$myWHITE"
-  ;;
-  *)
-    mySTATUS="$myGREEN$mySTATUS$myWHITE"
-  ;;
-esac
-
-printf "$mySTATUS"
-}
-
-function fuCONTAINERPORTS {
-local myNAME="$1"
-local myPORTS="$(/usr/bin/docker ps -f name=$myNAME --format "table {{.Ports}}" -f status=running -f status=exited | tail -n 1 | sed s/","/",\n\t\t\t\t\t\t\t"/g)"
-
-if [ "$myPORTS" != "PORTS" ];
-  then
-    printf "$myBLUE$myPORTS$myWHITE"
-fi
+function fuGETSTATUS {
+grc docker ps -f status=running -f status=exited --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}" | grep -v "NAME" | sort
 }
 
 function fuGETSYS {
@@ -51,16 +22,20 @@ echo
 
 while true
   do
+    myDPS=$(fuGETSTATUS)
+    myDPSNAMES=$(echo "$myDPS" | awk '{ print $1 }' | sort)
     fuGETSYS
-    printf "%-19s %-36s %s\n" "NAME" "STATUS" "PORTS"
-    for i in $myIMAGES; do
-          myNAME="$myMAGENTA$i$myWHITE"
-          printf "%-32s %-49s %s" "$myNAME" "$(fuCONTAINERSTATUS $i)" "$(fuCONTAINERPORTS $i)"
-          echo
-      if [ "$myPARAM" = "vv" ]; 
-        then
-          /usr/bin/docker exec -t "$i" /bin/ps awfuwfxwf | egrep -v -E "awfuwfxwf|/bin/ps" 
-      fi      
+    printf "%-21s %-28s %s\n" "NAME" "STATUS" "PORTS"
+    if [ "$myDPS" != "" ];
+      then
+        echo "$myDPS"
+    fi
+    for i in $myCONTAINERS; do
+      myAVAIL=$(echo "$myDPSNAMES" | grep -o "$i" | uniq | wc -l)      	    
+      if [ "$myAVAIL" = "0" ];
+	then
+	  printf "%-28s %-28s\n" "$myRED$i" "DOWN$myWHITE"
+      fi
     done
     if [[ $myPARAM =~ ^([1-9]|[1-9][0-9]|[1-9][0-9][0-9])$ ]];
       then 

bin/dump_es.sh

@@ -20,7 +20,7 @@ trap fuCLEANUP EXIT
 
 # Set vars
 myDATE=$(date +%Y%m%d%H%M)
-myINDICES=$(curl -s -XGET ''$myES'_cat/indices/' | grep logstash | awk '{ print $3 }' | sort | grep -v 1970)
+myINDICES=$(curl -s -XGET ''$myES'_cat/indices/' | awk '{ print $3 }' | sort | grep -v 1970)
 myES="http://127.0.0.1:64298/"
 myCOL1=""
 myCOL0=""
@@ -41,5 +41,5 @@ for i in $myINDICES;
 
 # Build tar archive
 echo $myCOL1"### Now building tar archive: es_dump_"$myDATE".tgz" $myCOL0
-tar cvf es_dump_$myDATE.tar tmp/*
+tar cvf es_dump_$myDATE.tar tmp/.
 echo $myCOL1"### Done."$myCOL0

bin/export_kibana-objects.sh

@@ -1,7 +1,8 @@
 #!/bin/bash
-# Export all Kibana objects
+# Export all Kibana objects through Kibana Saved Objects API
 # Make sure ES is available
 myES="http://127.0.0.1:64298/"
+myKIBANA="http://127.0.0.1:64296/"
 myESSTATUS=$(curl -s -XGET ''$myES'_cluster/health' | jq '.' | grep -c green)
 if ! [ "$myESSTATUS" = "1" ]
   then
@@ -14,10 +15,11 @@ fi
 
 # Set vars
 myDATE=$(date +%Y%m%d%H%M)
-myINDEXCOUNT=$(curl -s -XGET ''$myES'.kibana/index-pattern/logstash-*' | tr '\\' '\n' | grep "scripted" | wc -w)
-myDASHBOARDS=$(curl -s -XGET ''$myES'.kibana/dashboard/_search?filter_path=hits.hits._id&pretty&size=10000'  | jq '.hits.hits[] | {_id}' | jq -r '._id')
-myVISUALIZATIONS=$(curl -s -XGET ''$myES'.kibana/visualization/_search?filter_path=hits.hits._id&pretty&size=10000' | jq '.hits.hits[] | {_id}' | jq -r '._id')
-mySEARCHES=$(curl -s -XGET ''$myES'.kibana/search/_search?filter_path=hits.hits._id&pretty&size=10000' | jq '.hits.hits[] | {_id}' | jq -r '._id')
+myINDEXCOUNT=$(curl -s -XGET ''$myKIBANA'api/saved_objects/_find?type=index-pattern' | jq '.saved_objects[].attributes' | tr '\\' '\n' | grep "scripted" | wc -w)
+myINDEXID=$(curl -s -XGET ''$myKIBANA'api/saved_objects/_find?type=index-pattern' | jq '.saved_objects[].id' | tr -d '"')
+myDASHBOARDS=$(curl -s -XGET ''$myKIBANA'api/saved_objects/_find?type=dashboard&per_page=300' | jq '.saved_objects[].id' | tr -d '"')
+myVISUALIZATIONS=$(curl -s -XGET ''$myKIBANA'api/saved_objects/_find?type=visualization&per_page=300' | jq '.saved_objects[].id' | tr -d '"')
+mySEARCHES=$(curl -s -XGET ''$myKIBANA'api/saved_objects/_find?type=search&per_page=300' | jq '.saved_objects[].id' | tr -d '"')
 myCOL1=""
 myCOL0=""
 
@@ -29,8 +31,8 @@ trap fuCLEANUP EXIT
 
 # Export index patterns
 mkdir -p patterns
-echo $myCOL1"### Now exporting"$myCOL0 $myINDEXCOUNT $myCOL1"index patterns." $myCOL0
-curl -s -XGET ''$myES'.kibana/index-pattern/logstash-*?' | jq '._source' > patterns/index-patterns.json
+echo $myCOL1"### Now exporting"$myCOL0 $myINDEXCOUNT $myCOL1"index pattern fields." $myCOL0
+curl -s -XGET ''$myKIBANA'api/saved_objects/index-pattern/'$myINDEXID'' | jq '. | {attributes}' > patterns/$myINDEXID.json &
 echo
 
 # Export dashboards
@@ -39,7 +41,7 @@ echo $myCOL1"### Now exporting"$myCOL0 $(echo $myDASHBOARDS | wc -w) $myCOL1"das
 for i in $myDASHBOARDS;
   do
     echo $myCOL1"###### "$i $myCOL0
-    curl -s -XGET ''$myES'.kibana/dashboard/'$i'' | jq '._source' > dashboards/$i.json
+    curl -s -XGET ''$myKIBANA'api/saved_objects/dashboard/'$i'' | jq '. | {attributes}' > dashboards/$i.json &
   done;
 echo
 
@@ -49,7 +51,7 @@ echo $myCOL1"### Now exporting"$myCOL0 $(echo $myVISUALIZATIONS | wc -w) $myCOL1
 for i in $myVISUALIZATIONS;
   do
     echo $myCOL1"###### "$i $myCOL0
-    curl -s -XGET ''$myES'.kibana/visualization/'$i'' | jq '._source' > visualizations/$i.json
+    curl -s -XGET ''$myKIBANA'api/saved_objects/visualization/'$i'' | jq '. | {attributes}' > visualizations/$i.json &
   done;
 echo
 
@@ -59,10 +61,13 @@ echo $myCOL1"### Now exporting"$myCOL0 $(echo $mySEARCHES | wc -w) $myCOL1"searc
 for i in $mySEARCHES;
   do
     echo $myCOL1"###### "$i $myCOL0
-    curl -s -XGET ''$myES'.kibana/search/'$i'' | jq '._source' > searches/$i.json
+    curl -s -XGET ''$myKIBANA'api/saved_objects/search/'$i'' | jq '. | {attributes}' > searches/$i.json &
   done;
 echo
 
+# Wait for background exports to finish
+wait
+
 # Building tar archive
 echo $myCOL1"### Now building archive"$myCOL0 "kibana-objects_"$myDATE".tgz"
 tar cvfz kibana-objects_$myDATE.tgz patterns dashboards visualizations searches > /dev/null

bin/import_kibana-objects.sh

@@ -2,6 +2,7 @@
 # Import Kibana objects
 # Make sure ES is available
 myES="http://127.0.0.1:64298/"
+myKIBANA="http://127.0.0.1:64296/"
 myESSTATUS=$(curl -s -XGET ''$myES'_cluster/health' | jq '.' | grep -c green)
 if ! [ "$myESSTATUS" = "1" ]
   then
@@ -27,7 +28,7 @@ trap fuCLEANUP EXIT
 if [ "$myDUMP" = "" ];
   then
     echo $myCOL1"### Please provide a backup file name."$myCOL0 
-    echo $myCOL1"### restore-kibana-objects.sh <kibana-objects.tgz>"$myCOL0
+    echo $myCOL1"### import_kibana-objects.sh <kibana-objects.tgz>"$myCOL0
     echo 
     exit
 fi
@@ -41,10 +42,12 @@ fi
 tar xvfz $myDUMP > /dev/null
 
 # Restore index patterns
-myINDEXCOUNT=$(cat patterns/index-patterns.json | tr '\\' '\n' | grep "scripted" | wc -w)
-echo $myCOL1"### Now importing"$myCOL0 $myINDEXCOUNT $myCOL1"index patterns." $myCOL0
-curl -s -XDELETE ''$myES'.kibana/index-pattern/logstash-*' > /dev/null
-curl -s -XPUT ''$myES'.kibana/index-pattern/logstash-*' -T patterns/index-patterns.json > /dev/null
+myINDEXID=$(ls patterns/*.json | cut -c 10- | rev | cut -c 6- | rev)
+myINDEXCOUNT=$(cat patterns/$myINDEXID.json | tr '\\' '\n' | grep "scripted" | wc -w)
+echo $myCOL1"### Now importing"$myCOL0 $myINDEXCOUNT $myCOL1"index pattern fields." $myCOL0
+curl -s -XDELETE ''$myKIBANA'api/saved_objects/index-pattern/logstash-*' -H "Content-Type: application/json" -H "kbn-xsrf: true" > /dev/null
+curl -s -XDELETE ''$myKIBANA'api/saved_objects/index-pattern/'$myINDEXID'' -H "Content-Type: application/json" -H "kbn-xsrf: true" > /dev/null
+curl -s -XPOST ''$myKIBANA'api/saved_objects/index-pattern/'$myINDEXID'' -H "Content-Type: application/json" -H "kbn-xsrf: true" -d @patterns/$myINDEXID.json > /dev/null &
 echo
 
 # Restore dashboards
@@ -52,10 +55,15 @@ myDASHBOARDS=$(ls dashboards/*.json | cut -c 12- | rev | cut -c 6- | rev)
 echo $myCOL1"### Now importing "$myCOL0$(echo $myDASHBOARDS | wc -w)$myCOL1 "dashboards." $myCOL0
 for i in $myDASHBOARDS;
   do
-    echo $myCOL1"###### "$i $myCOL0
-    curl -s -XDELETE ''$myES'.kibana/dashboard/'$i'' > /dev/null
-    curl -s -XPUT ''$myES'.kibana/dashboard/'$i'' -T dashboards/$i.json > /dev/null
+    curl -s -XDELETE ''$myKIBANA'api/saved_objects/dashboard/'$i'' -H "Content-Type: application/json" -H "kbn-xsrf: true" > /dev/null &
   done;
+wait
+for i in $myDASHBOARDS;
+  do
+    echo $myCOL1"###### "$i $myCOL0
+    curl -s -XPOST ''$myKIBANA'api/saved_objects/dashboard/'$i'' -H "Content-Type: application/json" -H "kbn-xsrf: true" -d @dashboards/$i.json > /dev/null &
+  done;
+wait
 echo
 
 # Restore visualizations
@@ -63,22 +71,32 @@ myVISUALIZATIONS=$(ls visualizations/*.json | cut -c 16- | rev | cut -c 6- | rev
 echo $myCOL1"### Now importing "$myCOL0$(echo $myVISUALIZATIONS | wc -w)$myCOL1 "visualizations." $myCOL0
 for i in $myVISUALIZATIONS;
   do
-    echo $myCOL1"###### "$i $myCOL0
-    curl -s -XDELETE ''$myES'.kibana/visualization/'$i'' > /dev/null
-    curl -s -XPUT ''$myES'.kibana/visualization/'$i'' -T visualizations/$i.json > /dev/null
+    curl -s -XDELETE ''$myKIBANA'api/saved_objects/visualization/'$i'' -H "Content-Type: application/json" -H "kbn-xsrf: true" > /dev/null &
   done;
+wait
+for i in $myVISUALIZATIONS;
+  do
+    echo $myCOL1"###### "$i $myCOL0
+    curl -s -XPOST ''$myKIBANA'api/saved_objects/visualization/'$i'' -H "Content-Type: application/json" -H "kbn-xsrf: true" -d @visualizations/$i.json > /dev/null &
+  done;
+wait
 echo
 
 # Restore searches
 mySEARCHES=$(ls searches/*.json | cut -c 10- | rev | cut -c 6- | rev) 
 echo $myCOL1"### Now importing "$myCOL0$(echo $mySEARCHES | wc -w)$myCOL1 "searches." $myCOL0
+for i in $mySEARCHES;
+  do
+    curl -s -XDELETE ''$myKIBANA'api/saved_objects/search/'$i'' -H "Content-Type: application/json" -H "kbn-xsrf: true" > /dev/null &
+  done;
+wait
 for i in $mySEARCHES;
   do
     echo $myCOL1"###### "$i $myCOL0
-    curl -s -XDELETE ''$myES'.kibana/search/'$i'' > /dev/null
-    curl -s -XPUT ''$myES'.kibana/search/'$i'' -T searches/$i.json > /dev/null
+    curl -s -XPOST ''$myKIBANA'api/saved_objects/search/'$i'' -H "Content-Type: application/json" -H "kbn-xsrf: true" -d @searches/$i.json > /dev/null &
   done;
 echo
+wait
 
 # Stats
 echo

bin/restore_es.sh

@@ -42,6 +42,7 @@ tar xvf $myDUMP
 
 # Build indices list
 myINDICES=$(ls tmp/logstash*.gz | cut -c 5- | rev | cut -c 4- | rev)
+myINDICES+=" .kibana"
 echo $myCOL1"### The following indices will be restored: "$myCOL0
 echo $myINDICES
 echo

bin/rules.sh

@@ -0,0 +1,107 @@
+#!/bin/bash
+
+### Vars, Ports for Standard services
+myHOSTPORTS="7634 64294 64295"
+myDOCKERCOMPOSEYML="$1"
+myRULESFUNCTION="$2"
+
+function fuCHECKFORARGS {
+### Check if args are present, if not throw error
+
+if [ "$myDOCKERCOMPOSEYML" != "" ] && ([ "$myRULESFUNCTION" == "set" ] || [ "$myRULESFUNCTION" == "unset" ]);
+  then
+    echo "All arguments met. Continuing."
+  else
+    echo "Usage: rules.sh <docker-compose.yml> <[set, unset]>"
+    exit
+fi
+}
+
+function fuNFQCHECK {
+### Check if honeytrap or glutton is actively enabled in docker-compose.yml
+
+myNFQCHECK=$(grep -e '^\s*honeytrap:\|^\s*glutton:' $myDOCKERCOMPOSEYML | tr -d ': ' | uniq)
+if [ "$myNFQCHECK" == "" ];
+  then
+    echo "No NFQ related honeypot detected, no iptables rules needed. Exiting."
+    exit
+  else
+    echo "Detected $myNFQCHECK as NFQ based honeypot, iptables rules needed. Continuing."
+fi
+}
+
+function fuGETPORTS {
+### Get ports from docker-compose.yml
+
+myDOCKERCOMPOSEPORTS=$(cat $myDOCKERCOMPOSEYML | yq -r '.services[].ports' | grep ':' | sed -e s/127.0.0.1// | tr -d '", ' | sed -e s/^:// | cut -f1 -d ':' )
+myDOCKERCOMPOSEPORTS+=" $myHOSTPORTS"
+myRULESPORTS=$(for i in $myDOCKERCOMPOSEPORTS; do echo $i; done | sort -gu)
+echo "Setting up / removing these ports:"
+echo "$myRULESPORTS"
+}
+
+function fuSETRULES {
+### Setting up iptables rules for honeytrap
+if [ "$myNFQCHECK" == "honeytrap" ];
+  then
+    /sbin/iptables -w -A INPUT -s 127.0.0.1 -j ACCEPT
+    /sbin/iptables -w -A INPUT -d 127.0.0.1 -j ACCEPT
+
+    for myPORT in $myRULESPORTS; do
+      /sbin/iptables -w -A INPUT -p tcp --dport $myPORT -j ACCEPT
+    done
+
+    /sbin/iptables -w -A INPUT -p tcp --syn -m state --state NEW -j NFQUEUE
+fi
+
+### Setting up iptables rules for glutton
+if [ "$myNFQCHECK" == "glutton" ];
+  then
+    /sbin/iptables -w -t raw -A PREROUTING -s 127.0.0.1 -j ACCEPT
+    /sbin/iptables -w -t raw -A PREROUTING -d 127.0.0.1 -j ACCEPT
+
+    for myPORT in $myRULESPORTS; do
+      /sbin/iptables -w -t raw -A PREROUTING -p tcp --dport $myPORT -j ACCEPT
+    done
+    # No need for NFQ forwarding, such rules are set up by glutton
+fi
+}
+
+function fuUNSETRULES {
+### Removing  iptables rules for honeytrap
+if [ "$myNFQCHECK" == "honeytrap" ];
+  then
+    /sbin/iptables -w -D INPUT -s 127.0.0.1 -j ACCEPT
+    /sbin/iptables -w -D INPUT -d 127.0.0.1 -j ACCEPT
+
+    for myPORT in $myRULESPORTS; do
+      /sbin/iptables -w -D INPUT -p tcp --dport $myPORT -j ACCEPT
+    done
+
+    /sbin/iptables -w -D INPUT -p tcp --syn -m state --state NEW -j NFQUEUE
+fi
+
+### Removing iptables rules for glutton
+if [ "$myNFQCHECK" == "glutton" ];
+  then
+    /sbin/iptables -w -t raw -D PREROUTING -s 127.0.0.1 -j ACCEPT
+    /sbin/iptables -w -t raw -D PREROUTING -d 127.0.0.1 -j ACCEPT
+
+    for myPORT in $myRULESPORTS; do
+      /sbin/iptables -w -t raw -D PREROUTING -p tcp --dport $myPORT -j ACCEPT
+    done
+    # No need for removing NFQ forwarding, such rules are removed by glutton
+fi
+}
+
+# Main
+fuCHECKFORARGS
+fuNFQCHECK
+fuGETPORTS
+
+if [ "$myRULESFUNCTION" == "set" ];
+  then
+    fuSETRULES
+  else
+    fuUNSETRULES
+fi

bin/tped.sh

@@ -0,0 +1,48 @@
+#!/bin/bash
+
+# set backtitle, get filename
+myBACKTITLE="T-Pot Edition Selection Tool"
+myYMLS=$(cd /opt/tpot/etc/compose/ && ls -1 *.yml)
+myLINK="/opt/tpot/etc/tpot.yml"
+
+# Let's load docker images in parallel
+function fuPULLIMAGES {
+local myTPOTCOMPOSE="/opt/tpot/etc/tpot.yml"
+for name in $(cat $myTPOTCOMPOSE | grep -v '#' | grep image | cut -d'"' -f2 | uniq)
+  do
+    docker pull $name &
+  done
+wait
+echo
+}
+
+# setup menu
+for i in $myYMLS;
+  do
+    myITEMS+="$i $(echo $i | cut -d "." -f1 | tr [:lower:] [:upper:]) " 
+done
+myEDITION=$(dialog --backtitle "$myBACKTITLE" --menu "Select T-Pot Edition" 13 50 6 $myITEMS 3>&1 1>&2 2>&3 3>&-)
+if [ "$myEDITION" == "" ];
+  then
+    echo "Have a nice day!"
+    exit
+fi
+dialog --backtitle "$myBACKTITLE" --title "[ Activate now? ]" --yesno "\n$myEDITION" 7 50
+myOK=$?
+if [ "$myOK" == "0" ];
+  then
+    echo "OK - Activating and downloading latest images."
+    systemctl stop tpot
+    if [ "$(docker ps -aq)" != "" ];
+      then
+        docker stop $(docker ps -aq)
+        docker rm $(docker ps -aq)
+    fi
+    rm -f $myLINK
+    ln -s /opt/tpot/etc/compose/$myEDITION $myLINK
+    fuPULLIMAGES
+    systemctl start tpot
+    echo "Done. Use \"dps.sh\" for monitoring"
+  else 
+    echo "Have a nice day!"
+fi

bin/updateip.sh

@@ -8,9 +8,11 @@ if [ "$myEXTIP" = "" ];
   then
     myEXTIP=$myLOCALIP
 fi
+mySSHUSER=$(cat /etc/passwd | grep 1000 | cut -d ':' -f1)
 sed -i "s#IP:.*#IP: $myLOCALIP ($myEXTIP)#" /etc/issue
 sed -i "s#SSH:.*#SSH: ssh -l tsec -p 64295 $myLOCALIP#" /etc/issue
 sed -i "s#WEB:.*#WEB: https://$myLOCALIP:64297#" /etc/issue
+sed -i "s#ADMIN:.*#ADMIN: https://$myLOCALIP:64294#" /etc/issue
 tee /data/ews/conf/ews.ip << EOF
 [MAIN]
 ip = $myEXTIP

docker/ciscoasa/Dockerfile

@@ -0,0 +1,44 @@
+FROM alpine
+
+# Include dist
+ADD dist/ /root/dist/
+
+# Setup env and apt
+RUN apk -U upgrade && \
+    apk add build-base \
+            git \
+            libffi \
+            libffi-dev \
+            openssl \
+            openssl-dev \
+            python3 \
+            python3-dev && \
+
+# Setup user
+    addgroup -g 2000 ciscoasa && \
+    adduser -S -s /bin/bash -u 2000 -D -g 2000 ciscoasa && \
+
+# Get and install packages
+    mkdir -p /opt/ && \
+    cd /opt/ && \
+    git clone --depth=1 https://github.com/cymmetria/ciscoasa_honeypot && \
+    cd ciscoasa_honeypot && \
+    pip3 install --no-cache-dir --upgrade pip && \
+    pip3 install --no-cache-dir -r requirements.txt && \
+    cp /root/dist/asa_server.py /opt/ciscoasa_honeypot && \
+    chown -R ciscoasa:ciscoasa /opt/ciscoasa_honeypot && \
+
+# Clean up
+    apk del --purge build-base \
+                    git \
+                    libffi-dev \
+                    openssl-dev \
+                    python3-dev && \
+    rm -rf /root/* && \
+    rm -rf /var/cache/apk/*
+
+# Start ciscoasa
+STOPSIGNAL SIGINT
+WORKDIR /tmp/ciscoasa/
+USER ciscoasa:ciscoasa
+CMD cp -R /opt/ciscoasa_honeypot/* /tmp/ciscoasa && exec python3 asa_server.py --ike-port 5000 --enable_ssl --port 8443 --verbose >> /var/log/ciscoasa/ciscoasa.log 2>&1

docker/ciscoasa/README.md

@@ -0,0 +1,15 @@
+[![](https://images.microbadger.com/badges/version/dtagdevsec/ciscoasa:1811.svg)](https://microbadger.com/images/dtagdevsec/ciscoasa:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/ciscoasa:1811.svg)](https://microbadger.com/images/dtagdevsec/ciscoasa:1811 "Get your own image badge on microbadger.com")
+
+# ciscoasa
+
+[Ciscoasa](https://github.com/cymmetria/ciscoasa_honeypot) is a low interaction honeypot for the Cisco ASA component capable of detecting CVE-2018-0101, a DoS and remote code execution vulnerability
+
+This dockerized version is part of the **[T-Pot community honeypot](http://dtag-dev-sec.github.io/)** of Deutsche Telekom AG.
+
+The `Dockerfile` contains the blueprint for the dockerized ciscoasa and will be used to setup the docker image.
+
+The `docker-compose.yml` contains the necessary settings to test conpot using `docker-compose`. This will ensure to start the docker container with the appropriate permissions and port mappings.
+
+# Ciscoasa Dashboard
+
+![Ciscoasa Dashboard](doc/dashboard.png)

docker/ciscoasa/dist/asa_server.py

@@ -0,0 +1,307 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+import os
+import time
+import socket
+import logging
+logging.basicConfig(format='%(message)s')
+import threading
+from io import BytesIO
+from xml.etree import ElementTree
+from http.server import HTTPServer
+from socketserver import ThreadingMixIn
+from http.server import SimpleHTTPRequestHandler
+import ike_server
+import datetime
+
+
+class NonBlockingHTTPServer(ThreadingMixIn, HTTPServer):
+    pass
+
+class hpflogger:
+    def __init__(self, hpfserver, hpfport, hpfident, hpfsecret, hpfchannel, serverid, verbose):
+        self.hpfserver=hpfserver
+        self.hpfport=hpfport
+        self.hpfident=hpfident
+        self.hpfsecret=hpfsecret
+        self.hpfchannel=hpfchannel
+        self.serverid=serverid
+        self.hpc=None
+        self.verbose=verbose
+        if (self.hpfserver and self.hpfport and self.hpfident and self.hpfport and self.hpfchannel and self.serverid):
+            import hpfeeds
+            try:
+                self.hpc = hpfeeds.new(self.hpfserver, self.hpfport, self.hpfident, self.hpfsecret)
+                logger.debug("Logging to hpfeeds using server: {0}, channel {1}.".format(self.hpfserver, self.hpfchannel))
+            except (hpfeeds.FeedException, socket.error, hpfeeds.Disconnect):
+                logger.critical("hpfeeds connection not successful")
+
+    def log(self, level, message):
+        if self.hpc:
+            if level in ['debug', 'info'] and not self.verbose:
+                return
+            self.hpc.publish(self.hpfchannel, "["+self.serverid+"] ["+level+"] ["+datetime.datetime.now().isoformat() +"] "  + str(message))
+
+
+def header_split(h):
+    return [list(map(str.strip, l.split(': ', 1))) for l in h.strip().splitlines()]
+
+
+class WebLogicHandler(SimpleHTTPRequestHandler):
+    logger = None
+    hpfl = None
+
+    protocol_version = "HTTP/1.1"
+
+    EXPLOIT_STRING = b"host-scan-reply"
+    RESPONSE = b"""<?xml version="1.0" encoding="UTF-8"?>
+<config-auth client="vpn" type="complete">
+<version who="sg">9.0(1)</version>
+<error id="98" param1="" param2="">VPN Server could not parse request.</error>
+</config-auth>"""
+
+    basepath = os.path.dirname(os.path.abspath(__file__))
+
+    alert_function = None
+
+    def setup(self):
+        SimpleHTTPRequestHandler.setup(self)
+        self.request.settimeout(3)
+
+    def send_header(self, keyword, value):
+        if keyword.lower() == 'server':
+            return
+        SimpleHTTPRequestHandler.send_header(self, keyword, value)
+
+    def send_head(self):
+        # send_head will return a file object that do_HEAD/GET will use
+        # do_GET/HEAD are already implemented by SimpleHTTPRequestHandler
+        filename = os.path.basename(self.path.rstrip('/').split('?', 1)[0])
+
+        if self.path == '/':
+            self.send_response(200)
+            for k, v in header_split("""
+                Content-Type: text/html
+                Cache-Control: no-cache
+                Pragma: no-cache
+                Set-Cookie: tg=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+                Set-Cookie: webvpn=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+                Set-Cookie: webvpnc=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+                Set-Cookie: webvpn_portal=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+                Set-Cookie: webvpnSharePoint=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+                Set-Cookie: webvpnlogin=1; path=/; secure
+                Set-Cookie: sdesktop=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+            """):
+                self.send_header(k, v)
+            self.end_headers()
+            return BytesIO(b'<html><script>document.location.replace("/+CSCOE+/logon.html")</script></html>\n')
+        elif filename == 'asa':  # don't allow dir listing
+            return self.send_file('wrong_url.html', 403)
+        else:
+            return self.send_file(filename)
+
+    def redirect(self, loc):
+        self.send_response(302)
+        for k, v in header_split("""
+            Content-Type: text/html
+            Content-Length: 0
+            Cache-Control: no-cache
+            Pragma: no-cache
+            Location: %s
+            Set-Cookie: tg=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+        """ % (loc,)):
+            self.send_header(k, v)
+        self.end_headers()
+
+    def do_GET(self):
+        if self.path == '/+CSCOE+/logon.html':
+            self.redirect('/+CSCOE+/logon.html?fcadbadd=1')
+            return
+        elif self.path.startswith('/+CSCOE+/logon.html?') and 'reason=1' in self.path:
+            self.wfile.write(self.send_file('logon_failure').getvalue())
+            return
+        SimpleHTTPRequestHandler.do_GET(self)
+
+    def do_POST(self):
+        data_len = int(self.headers.get('Content-length', 0))
+        data = self.rfile.read(data_len) if data_len else b''
+        body = self.RESPONSE
+        if self.EXPLOIT_STRING in data:
+            xml = ElementTree.fromstring(data)
+            payloads = []
+            for x in xml.iter('host-scan-reply'):
+                payloads.append(x.text)
+
+            self.alert_function(self.client_address[0], self.client_address[1], payloads)
+
+        elif self.path == '/':
+            self.redirect('/+webvpn+/index.html')
+            return
+        elif self.path == '/+CSCOE+/logon.html':
+            self.redirect('/+CSCOE+/logon.html?fcadbadd=1')
+            return
+        elif self.path.split('?', 1)[0] == '/+webvpn+/index.html':
+            with open(os.path.join(self.basepath, 'asa', "logon_redir.html"), 'rb') as fh:
+                body = fh.read()
+
+        self.send_response(200)
+        self.send_header('Content-Length', int(len(body)))
+        self.send_header('Content-Type', 'text/html; charset=UTF-8')
+        self.end_headers()
+        self.wfile.write(body)
+        return
+
+    def send_file(self, filename, status_code=200, headers=[]):
+        try:
+            with open(os.path.join(self.basepath, 'asa', filename), 'rb') as fh:
+                body = fh.read()
+                self.send_response(status_code)
+                for k, v in headers:
+                    self.send_header(k, v)
+                if status_code == 200:
+                    for k, v in header_split("""
+                        Cache-Control: max-age=0
+                        Set-Cookie: webvpn=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+                        Set-Cookie: webvpnc=; expires=Thu, 01 Jan 1970 22:00:00 GMT; path=/; secure
+                        Set-Cookie: webvpnlogin=1; secure
+                        X-Transcend-Version: 1
+                    """):
+                        self.send_header(k, v)
+                self.send_header('Content-Length', int(len(body)))
+                self.send_header('Content-Type', 'text/html')
+                self.end_headers()
+                return BytesIO(body)
+        except IOError:
+            return self.send_file('wrong_url.html', 404)
+
+    def log_message(self, format, *args):
+        self.logger.debug("{'timestamp': '%s', 'src_ip': '%s', 'payload_printable': '%s'}" %
+                          (datetime.datetime.now().isoformat(),
+                           self.client_address[0],
+                           format % args))
+        self.hpfl.log('debug', "%s - - [%s] %s" %
+                          (self.client_address[0],
+                           self.log_date_time_string(),
+                           format % args))
+
+    def handle_one_request(self):
+        """Handle a single HTTP request.
+        Overriden to not send 501 errors
+        """
+        self.close_connection = True
+        try:
+            self.raw_requestline = self.rfile.readline(65537)
+            if len(self.raw_requestline) > 65536:
+                self.requestline = ''
+                self.request_version = ''
+                self.command = ''
+                self.close_connection = 1
+                return
+            if not self.raw_requestline:
+                self.close_connection = 1
+                return
+            if not self.parse_request():
+                # An error code has been sent, just exit
+                return
+            mname = 'do_' + self.command
+            if not hasattr(self, mname):
+                self.log_request()
+                self.close_connection = True
+                return
+            method = getattr(self, mname)
+            method()
+            self.wfile.flush()  # actually send the response if not already done.
+        except socket.timeout as e:
+            # a read or a write timed out.  Discard this connection
+            self.log_error("Request timed out: %r", e)
+            self.close_connection = 1
+            return
+
+
+if __name__ == '__main__':
+    import click
+
+    logging.basicConfig(level=logging.INFO)
+    logger = logging.getLogger()
+    logger.info('info')
+
+    @click.command()
+    @click.option('-h', '--host', default='0.0.0.0', help='Host to listen')
+    @click.option('-p', '--port', default=8443, help='Port to listen', type=click.INT)
+    @click.option('-i', '--ike-port', default=5000, help='Port to listen for IKE', type=click.INT)
+    @click.option('-s', '--enable_ssl', default=False, help='Enable SSL', is_flag=True)
+    @click.option('-c', '--cert', default=None, help='Certificate File Path (will generate self signed '
+                                                     'cert if not supplied)')
+    @click.option('-v', '--verbose', default=False, help='Verbose logging', is_flag=True)
+
+    # hpfeeds options
+    @click.option('--hpfserver', default=os.environ.get('HPFEEDS_SERVER'), help='HPFeeds Server')
+    @click.option('--hpfport', default=os.environ.get('HPFEEDS_PORT'), help='HPFeeds Port', type=click.INT)
+    @click.option('--hpfident', default=os.environ.get('HPFEEDS_IDENT'), help='HPFeeds Ident')
+    @click.option('--hpfsecret', default=os.environ.get('HPFEEDS_SECRET'), help='HPFeeds Secret')
+    @click.option('--hpfchannel', default=os.environ.get('HPFEEDS_CHANNEL'), help='HPFeeds Channel')
+    @click.option('--serverid', default=os.environ.get('SERVERID'), help='Verbose logging')
+
+
+    def start(host, port, ike_port, enable_ssl, cert, verbose, hpfserver, hpfport, hpfident, hpfsecret, hpfchannel, serverid):
+        """
+           A low interaction honeypot for the Cisco ASA component capable of detecting CVE-2018-0101,
+           a DoS and remote code execution vulnerability
+        """
+
+        hpfl=hpflogger(hpfserver, hpfport, hpfident, hpfsecret, hpfchannel, serverid, verbose)
+
+        def alert(cls, host, port, payloads):
+            logger.critical({
+                 'timestamp': datetime.datetime.utcnow().isoformat(),
+                 'src_ip': host,
+                 'src_port': port,
+                 'payload_printable': payloads,
+            })
+            #log to hpfeeds
+            hpfl.log("critical", {
+                 'src': host,
+                 'spt': port,
+                 'data': payloads,
+             })
+
+        if verbose:
+            logger.setLevel(logging.DEBUG)
+
+        requestHandler = WebLogicHandler
+        requestHandler.alert_function = alert
+        requestHandler.logger = logger
+        requestHandler.hpfl = hpfl
+
+        def log_date_time_string():
+            """Return the current time formatted for logging."""
+            now = datetime.datetime.now().isoformat()
+            return now
+
+        def ike():
+            ike_server.start(host, ike_port, alert, logger, hpfl)
+        t = threading.Thread(target=ike)
+        t.daemon = True
+        t.start()
+
+        httpd = HTTPServer((host, port), requestHandler)
+        if enable_ssl:
+            import ssl
+            if not cert:
+                import gencert
+                cert = gencert.gencert()
+            httpd.socket = ssl.wrap_socket(httpd.socket, certfile=cert, server_side=True)
+
+        logger.info('Starting server on port {:d}/tcp, use <Ctrl-C> to stop'.format(port))
+        hpfl.log('info', 'Starting server on port {:d}/tcp, use <Ctrl-C> to stop'.format(port))
+
+        try:
+            httpd.serve_forever()
+        except KeyboardInterrupt:
+            pass
+        logger.info('Stopping server.')
+        hpfl.log('info', 'Stopping server.')
+
+        httpd.server_close()
+
+    start()

docker/ciscoasa/docker-compose.yml

@@ -0,0 +1,19 @@
+version: '2.3'
+
+services:
+
+# Ciscoasa service
+  ciscoasa:
+    build: .
+    container_name: ciscoasa
+    restart: always
+    tmpfs:
+     - /tmp/ciscoasa:uid=2000,gid=2000
+    network_mode: "host"
+    ports:
+     - "5000:5000/udp"
+     - "8443:8443"
+    image: "dtagdevsec/ciscoasa:1811"
+    read_only: true
+    volumes:
+     - /data/ciscoasa/log:/var/log/ciscoasa

docker/conpot/Dockerfile

@@ -1,43 +1,60 @@
 FROM alpine
-MAINTAINER MO
 
 # Include dist
 ADD dist/ /root/dist/
 
 # Setup apt
-RUN apk -U add bash \
-               build-base \
-               file \
-               git \
-               libev \
-               libtool \
-               libxslt \
-               libxslt-dev \
-               mariadb-dev \
-               mariadb-client-libs \
-               pkgconfig \
-               python \
-               python-dev \
-               py-cffi && \
+RUN apk -U add \
+             build-base \
+             file \
+             git \
+             libev \
+             libtool \
+             libcap \
+             libxslt \
+             libxslt-dev \
+             mariadb-dev \
+             pkgconfig \
+             python3 \
+             python3-dev \
+             py-cffi \
+             py-cryptography \
+             tcpdump \
+             wget && \
 
 # Setup ConPot
-    git clone https://github.com/mushorg/conpot /opt/conpot/ && \
+    git clone --depth=1 https://github.com/mushorg/conpot /opt/conpot && \
     cd /opt/conpot/ && \
-    git checkout d97a68a054e4fe42ff90293188a5702ce8ab09a3 && \
-    cp /root/dist/requirements.txt /opt/conpot/ && \
-    python setup.py install && \
+    # Patch to accept ENV for MIB path
+    sed -i "s/tmp_mib_dir = tempfile.mkdtemp()/tmp_mib_dir = tempfile.mkdtemp(dir=os.environ['CONPOT_TMP'])/" /opt/conpot/conpot/protocols/snmp/snmp_server.py && \
+    # Change template default ports if <1024
+    sed -i 's/port="2121"/port="21"/' /opt/conpot/conpot/templates/default/ftp/ftp.xml && \ 
+    sed -i 's/port="8800"/port="80"/' /opt/conpot/conpot/templates/default/http/http.xml && \ 
+    sed -i 's/port="6230"/port="623"/' /opt/conpot/conpot/templates/default/ipmi/ipmi.xml && \ 
+    sed -i 's/port="5020"/port="502"/' /opt/conpot/conpot/templates/default/modbus/modbus.xml && \ 
+    sed -i 's/port="10201"/port="102"/' /opt/conpot/conpot/templates/default/s7comm/s7comm.xml && \ 
+    sed -i 's/port="16100"/port="161"/' /opt/conpot/conpot/templates/default/snmp/snmp.xml && \ 
+    sed -i 's/port="6969"/port="69"/' /opt/conpot/conpot/templates/default/tftp/tftp.xml && \ 
+    sed -i 's/port="16100"/port="161"/' /opt/conpot/conpot/templates/IEC104/snmp/snmp.xml && \ 
+    sed -i 's/port="6230"/port="623"/' /opt/conpot/conpot/templates/ipmi/ipmi/ipmi.xml && \ 
+    pip3 install --no-cache-dir -U pip setuptools && \
+    pip3 install --no-cache-dir . && \
     cd / && \
     rm -rf /opt/conpot /tmp/* /var/tmp/* && \
-
-# Setup user, groups and configs
+    setcap cap_net_bind_service=+ep /usr/bin/python3.6 && \
+    
+# Get wireshark manuf db for scapy, setup configs, user, groups
+    mkdir -p /etc/conpot /var/log/conpot /usr/share/wireshark && \
+    wget https://github.com/wireshark/wireshark/raw/master/manuf -o /usr/share/wireshark/manuf && \
+    cp /root/dist/conpot.cfg /etc/conpot/conpot.cfg && \
+    cp -R /root/dist/templates /usr/lib/python3.6/site-packages/conpot/ && \
     addgroup -g 2000 conpot && \
-    adduser -S -s /bin/bash -u 2000 -D -g 2000 conpot && \
-    mkdir -p /etc/conpot /var/log/conpot && \
-    mv /root/dist/conpot.cfg /etc/conpot/conpot.cfg && \
-    mv /root/dist/kamstrup_382/template.xml /usr/lib/python2.7/site-packages/Conpot-0.5.1-py2.7.egg/conpot/templates/kamstrup_382/ && \
+    adduser -S -s /bin/ash -u 2000 -D -g 2000 conpot && \
 
 # Clean up
-    apk del build-base \
+    apk del --purge \
+            build-base \
+            cython-dev \
             file \
             git \
             libev \
@@ -45,10 +62,14 @@ RUN apk -U add bash \
             libxslt-dev \
             mariadb-dev \
             pkgconfig \
-            python-dev \
-            py-cffi && \
+            python3-dev \
+            py-cffi \
+            wget && \
     rm -rf /root/* && \
+    rm -rf /tmp/* && \
     rm -rf /var/cache/apk/*
 
-# Run supervisor upon container start
-CMD ["/usr/bin/conpot", "--template", "kamstrup_382", "--logfile", "/var/log/conpot/conpot.log", "--config", "/etc/conpot/conpot.cfg"]
+# Start conpot
+STOPSIGNAL SIGINT
+USER conpot:conpot
+CMD exec /usr/bin/conpot --temp_dir $CONPOT_TMP --template $CONPOT_TEMPLATE --logfile $CONPOT_LOG --config $CONPOT_CONFIG

docker/conpot/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/conpot:1710.svg)](https://microbadger.com/images/dtagdevsec/conpot:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/conpot:1710.svg)](https://microbadger.com/images/dtagdevsec/conpot:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/conpot:1811.svg)](https://microbadger.com/images/dtagdevsec/conpot:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/conpot:1811.svg)](https://microbadger.com/images/dtagdevsec/conpot:1811 "Get your own image badge on microbadger.com")
 
 # conpot
 
@@ -12,4 +12,4 @@ The `docker-compose.yml` contains the necessary settings to test conpot using `d
 
 # ConPot Dashboard
 
-![ConPot Dashboard](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/docker/conpot/doc/dashboard.png)
+![ConPot Dashboard](doc/dashboard.png)

docker/conpot/dist/conpot.cfg

@@ -1,6 +1,10 @@
 [common]
 sensorid = conpot
 
+[virtual_file_system]
+data_fs_url = %(CONPOT_TMP)s
+fs_url = tar:///usr/lib/python3.6/site-packages/conpot/data.tar
+
 [session]
 timeout = 30
 
@@ -10,7 +14,7 @@ group = conpot
 
 [json]
 enabled = True
-filename = /var/log/conpot/conpot.json
+filename = %(CONPOT_JSON_LOG)s
 
 [sqlite]
 enabled = False

docker/conpot/dist/requirements.txt

@@ -1,23 +0,0 @@
-gevent>=1.0
-pysnmp==4.3.5
-pysmi==0.1.3
-lxml
-bottle
-jinja2
-beautifulsoup4
-requests
-sphinx==1.5.5
-libtaxii>=1.1.0
-MySQL-python
-xlrd
-crc16
-enum
-hpfeeds
-modbus-tk
-stix-validator
-stix
-cybox
-bacpypes==0.13.8
-pyghmi
-mixbox
-modbus-tk

docker/conpot/dist/templates/IEC104/template.xml

@@ -0,0 +1,675 @@
+<!-- Copyright (C) 2017  Patrick Reichenberger (University of Passau) <patrick.reichenberger@t-online.de>
+
+ This program is free software; you can redistribute it and/or
+ modify it under the terms of the GNU General Public License
+ as published by the Free Software Foundation; either version 2
+ of the License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc.,
+ 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+-->
+
+<core>
+    <template>
+        <!-- General information about the template -->
+        <entity name="unit">S7-300</entity>
+        <entity name="vendor">Siemens</entity>
+        <entity name="description">Creates a simple device for IEC 60870-5-104</entity>
+        <entity name="protocols">IEC104, SNMP</entity>
+        <entity name="creator">Patrick Reichenberger</entity>
+    </template>
+    <databus>
+        <!-- Core value that can be retrieved from the databus by key -->
+        <key_value_mappings>
+            <!-- SNMPv2-MIB -->
+            <key name="SystemDescription">
+                <value type="value">"Siemens, SIMATIC, S7-300"</value>
+            </key>
+            <key name="sysObjectID">
+                <value type="value">"0.0"</value>
+            </key>
+            <key name="Uptime">
+                <value type="function">conpot.emulators.misc.uptime.Uptime</value>
+            </key>
+            <key name="sysContact">
+                <value type="value">"Corporate IT"</value>
+            </key>
+            <key name="sysName">
+                <value type="value">"DE-BER01"</value>
+            </key>
+            <key name="sysLocation">
+                <value type="value">"BER01, T2E"</value>
+            </key>
+            <key name="sysServices">
+                <value type="value">"72"</value>
+            </key>
+            <!-- IF-MIB -->
+            <key name="ifNumber">
+                <value type="value">1</value>
+            </key>
+            <key name="ifIndex">
+                <value type="value">1</value>
+            </key>
+            <key name="ifDescr">
+                <value type="value">"Siemens, SIMATIC NET, CP 343-1 PN, 6GK7 343-1EX21-0XE0, HW: Version 2, FW: Version V1.2.3, Ethernet Port 1, Rack 0, 100Mbit"</value>
+            </key>
+            <key name="ifType">
+                <value type="value">6</value>
+            </key>
+            <key name="ifMtu">
+                <value type="value">1000</value>
+            </key>
+            <key name="ifSpeed">
+                <value type="value">100000000</value>
+            </key>
+            <key name="ifPhysAddress">
+                <value type="value">"\x00\x0e\x8c\x29\xc5\x1a"</value>
+            </key>
+            <key name="ifAdminStatus">
+                <value type="value">1</value>
+            </key>
+            <key name="ifOperStatus">
+                <value type="value">1</value>
+            </key>
+            <key name="ifLastChange">
+                <value type="function">conpot.emulators.misc.uptime.Uptime</value>
+            </key>
+            <key name="FacilityName">
+                <value type="value">"Compagnie Generale des Eaux"</value>
+            </key>
+            <key name="0">
+                <value type="value">0</value>
+            </key>
+            <key name="1">
+                <value type="value">1</value>
+            </key>
+            <key name="ifInOctets">
+                <value type="value">1618895</value>
+            </key>
+            <key name="ifInUcastPkts">
+                <value type="value">7018</value>
+            </key>
+            <key name="ifInNUcastPkts">
+                <value type="value">291</value>
+            </key>
+            <key name="ifOutOctets">
+                <value type="value">455107</value>
+            </key>
+            <key name="ifOutUcastPkts">
+                <value type="value">872264</value>
+            </key>
+            <key name="ifOutUNcastPkts">
+                <value type="value">143</value>
+            </key>
+
+            <!-- IP-MIB -->
+            <key name="ipForwarding">
+                <value type="value">2</value>
+            </key>
+            <key name="ipDefaultTTL">
+                <value type="value">60</value>
+            </key>
+            <key name="ipInReceives">
+                <value type="value">31271</value>
+            </key>
+            <key name="ipInHdrErrors">
+                <value type="value">0</value>
+            </key>
+            <key name="ipInAddrErrors">
+                <value type="value">0</value>
+            </key>
+            <key name="ipForwDatagrams">
+                <value type="value">0</value>
+            </key>
+            <key name="ipInUnknownProtos">
+                <value type="value">0</value>
+            </key>
+            <key name="ipInDiscards">
+                <value type="value">0</value>
+            </key>
+            <key name="ipInDelivers">
+                <value type="value">31282</value>
+            </key>
+            <key name="ipOutRequests">
+                <value type="value">69023</value>
+            </key>
+            <key name="ipOutDiscards">
+                <value type="value">0</value>
+            </key>
+            <key name="ipOutNoRoutes">
+                <value type="value">0</value>
+            </key>
+            <key name="ipReasmTimeout">
+                <value type="value">60</value>
+            </key>
+            <key name="ipReasmReqds">
+                <value type="value">7</value>
+            </key>
+            <key name="ipReasmOKs">
+                <value type="value">3</value>
+            </key>
+            <key name="ipReasmFails">
+                <value type="value">0</value>
+            </key>
+            <key name="ipFragOKs">
+                <value type="value">0</value>
+            </key>
+            <key name="ipFragFails">
+                <value type="value">0</value>
+            </key>
+            <key name="ipFragCreates">
+                <value type="value">0</value>
+            </key>
+            <key name="ipAdEntAddr">
+                <value type="value">"217.172.190.137"</value>
+            </key>
+            <key name="ipAdEntIfIndex">
+                <value type="value">1</value>
+            </key>
+            <key name="ipAdEntNetMask">
+                <value type="value">"255.255.255.255"</value>
+            </key>
+            <key name="ipAdEntBcastAddr">
+                <value type="value">1</value>
+            </key>
+            <key name="ipAdEntReasmMaxSize">
+                <value type="value">65528</value>
+            </key>
+            <key name="ipRoutingDiscards">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInMsgs">
+                <value type="value">4</value>
+            </key>
+            <key name="icmpInErrors">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInDestUnreachs">
+                <value type="value">1</value>
+            </key>
+            <key name="icmpInTimeExcds">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInParmProbs">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInSrcQuenchs">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInRedirects">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInEchos">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInEchoReps">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInTimestamps">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInTimestampReps">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInAddrMasks">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpInAddrMaskReps">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutMsgs">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutErrors">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutDestUnreachs">
+                <value type="value">144</value>
+            </key>
+            <key name="icmpOutTimeExcds">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutParmProbs">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutSrcQuenchs">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutRedirects">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutEchos">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutEchoReps">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutTimestamps">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutTimestampReps">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutAddrMasks">
+                <value type="value">0</value>
+            </key>
+            <key name="icmpOutAddrMaskReps">
+                <value type="value">0</value>
+            </key>
+
+            <!-- TCP-MIB -->
+            <key name="tcpRtoAlgorithm">
+                <value type="value">2</value>
+            </key>
+            <key name="tcpRtoMin">
+                <value type="value">0</value>
+            </key>
+            <key name="tcpRtoMax">
+                <value type="value">100</value>
+            </key>
+            <key name="tcpMaxConn">
+                <value type="value">-1</value>
+            </key>
+            <key name="tcpActiveOpens">
+                <value type="value">0</value>
+            </key>
+            <key name="tcpPassiveOpens">
+                <value type="value">101</value>
+            </key>
+            <key name="tcpAttemptFails">
+                <value type="value">42</value>
+            </key>
+            <key name="tcpEstabResets">
+                <value type="value">45</value>
+            </key>
+            <key name="tcpCurrEstab">
+                <value type="value">0</value>
+            </key>
+            <key name="tcpInSegs">
+                <value type="value">30321</value>
+            </key>
+            <key name="tcpOutSegs">
+                <value type="value">67821</value>
+            </key>
+            <key name="tcpRetransSegs">
+                <value type="value">2511</value>
+            </key>
+            <key name="tcpConnState">
+                <value type="value">2</value>
+            </key>
+            <key name="tcpConnLocalAddress">
+                <value type="value">"217.172.190.137"</value>
+            </key>
+            <key name="tcpConnLocalPort">
+                <value type="value">2404</value>
+            </key>
+            <key name="tcpConnRemAddress">
+                <value type="value">"0.0.0.0"</value>
+            </key>
+            <key name="tcpConnRemPort">
+                <value type="value">0</value>
+            </key>
+            <key name="tcpInErrs">
+                <value type="value">1</value>
+            </key>
+            <key name="tcpOutRsts">
+                <value type="value">728</value>
+            </key>
+            <!-- UDP-MIB -->
+            <key name="udpInDatagrams">
+                <value type="value">1441</value>
+            </key>
+            <key name="udpNoPorts">
+                <value type="value">1280</value>
+            </key>
+            <key name="udpInErrors">
+                <value type="value">23</value>
+            </key>
+            <key name="udpOutDatagrams">
+                <value type="value">47</value>
+            </key>
+            <key name="udpLocalAddress">
+                <value type="value">"217.172.190.137"</value>
+            </key>
+            <key name="udpLocalPort">
+                <value type="value">161</value>
+            </key>
+            <key name="SystemName">
+                <value type="value">"CP 343-1 IT"</value>
+            </key>
+
+
+            <!-- IEC104 Protocol parameter -->
+            <!-- Timeout of connection establishment -->
+            <key name="T_0">
+                <value type="value">30</value>
+            </key>
+            <!-- Timeout of send or test APDUs (Wartezeit auf Quittung) -->
+            <key name="T_1">
+                <value type="value">15</value>
+            </key>
+            <!-- Timeout for acknowledges in case of no data messages T_2 < T_1 (Quittieren nach x sek) -->
+            <key name="T_2">
+                <value type="value">10</value>
+            </key>
+            <!-- Timeout for sending test frames in case of a long idle state -->
+            <key name="T_3">
+                <value type="value">20</value>
+            </key>
+            <!-- Maximum difference receive sequence number to send state variable (Max. Anzahl unquittierter Telegramme) -->
+            <!-- not implemented yet -->
+            <key name="k">
+                <value type="value">12</value>
+            </key>
+            <!-- Latest acknowledge after receiving w I-format APDUs (Quittieren nach w Telegrammen) -->
+            <key name="w">
+                <value type="value">8</value>
+            </key>
+            <!-- Maximum frame size (in bytes) -->
+            <key name="MaxFrameSize">
+                <value type="value">254</value>
+            </key>
+
+            <!-- Devices -->
+            <!-- 13- -->
+            <key name="13_20">
+                <value type="value">1</value>
+            </key>
+            <key name="13_21">
+                <value type="value">0</value>
+            </key>
+            <key name="13_22">
+                <value type="value">0</value>
+            </key>
+            <key name="13_24">
+                <value type="value">1</value>
+            </key>
+            <key name="13_25">
+                <value type="value">1</value>
+            </key>
+            <key name="13_32">
+                <value type="value">1</value>
+            </key>
+            <key name="13_33">
+                <value type="value">1</value>
+            </key>
+            <key name="13_34">
+                <value type="value">1</value>
+            </key>
+            <key name="13_35">
+                <value type="value">1</value>
+            </key>
+            <key name="13_36">
+                <value type="value">1</value>
+            </key>
+            <key name="13_37">
+                <value type="value">1</value>
+            </key>
+            <key name="13_38">
+                <value type="value">1</value>
+            </key>
+            <key name="13_39">
+                <value type="value">1</value>
+            </key>
+            <key name="13_40">
+                <value type="value">0</value>
+            </key>
+            <key name="13_41">
+                <value type="value">1</value>
+            </key>
+            <key name="13_42">
+                <value type="value">0</value>
+            </key>
+
+            <!-- 22- -->
+            <key name="22_19">
+                <value type="value">1</value>
+            </key>
+            <key name="22_20">
+                <value type="value">1</value>
+            </key>
+            <key name="22_21">
+                <value type="value">0</value>
+            </key>
+            <key name="22_22">
+                <value type="value">0</value>
+            </key>
+            <key name="22_24">
+                <value type="value">1</value>
+            </key>
+            <key name="22_25">
+                <value type="value">1</value>
+            </key>
+            <key name="22_42">
+                <value type="value">1</value>
+            </key>
+            <key name="22_43">
+                <value type="value">1</value>
+            </key>
+            <key name="22_54">
+                <value type="value">1</value>
+            </key>
+
+            <!-- 33- -->
+            <key name="33_2">
+                <value type="value">1</value>
+            </key>
+            <key name="33_3">
+                <value type="value">2</value>
+            </key>
+            <key name="33_4">
+                <value type="value">1</value>
+            </key>
+            <key name="33_5">
+                <value type="value">2</value>
+            </key>
+            <key name="33_6">
+                <value type="value">2</value>
+            </key>
+            <key name="33_7">
+                <value type="value">1</value>
+            </key>
+            <key name="33_8">
+                <value type="value">1</value>
+            </key>
+            <key name="33_9">
+                <value type="value">1</value>
+            </key>
+            <key name="33_10">
+                <value type="value">1</value>
+            </key>
+            <key name="33_11">
+                <value type="value">1</value>
+            </key>
+
+            <!-- 60- -->
+            <key name="60_6">
+                <value type="value">2</value>
+            </key>
+            <key name="60_7">
+                <value type="value">1</value>
+            </key>
+            <key name="60_8">
+                <value type="value">1</value>
+            </key>
+            <key name="60_9">
+                <value type="value">1</value>
+            </key>
+            <key name="60_20">
+                <value type="value">1</value>
+            </key>
+            <key name="60_21">
+                <value type="value">1</value>
+            </key>
+            <key name="60_32">
+                <value type="value">1</value>
+            </key>
+            <key name="60_34">
+                <value type="value">1</value>
+            </key>
+            <key name="60_35">
+                <value type="value">1</value>
+            </key>
+            <key name="60_36">
+                <value type="value">1</value>
+            </key>
+
+            <!-- 100- -->
+            <key name="100_12">
+                <value type="value">103</value>
+            </key>
+            <key name="100_13">
+                <value type="value">31</value>
+            </key>
+            <key name="100_51">
+                <value type="value">-49</value>
+            </key>
+            <key name="100_108">
+                <value type="value">28871</value>
+            </key>
+            <key name="100_109">
+                <value type="value">13781</value>
+            </key>
+            <key name="100_178">
+                <value type="value">119</value>
+            </key>
+            <key name="100_179">
+                <value type="value">219</value>
+            </key>
+            <key name="100_190">
+                <value type="value">1009</value>
+            </key>
+            <key name="100_191">
+                <value type="value">-2</value>
+            </key>
+            <key name="100_192">
+                <value type="value">701</value>
+            </key>
+            <key name="100_193">
+                <value type="value">441</value>
+            </key>
+
+            <!-- 101- -->
+            <key name="101_63">
+                <value type="value">103</value>
+            </key>
+            <key name="101_205">
+                <value type="value">31</value>
+            </key>
+            <key name="101_100">
+                <value type="value">5</value>
+            </key>
+            <key name="101_101">
+                <value type="value">49</value>
+            </key>
+            <key name="101_102">
+                <value type="value">119</value>
+            </key>
+            <key name="101_105">
+                <value type="value">500</value>
+            </key>
+            <key name="101_106">
+                <value type="value">1</value>
+            </key>
+
+            <!-- 107- -->
+            <key name="107_3">
+                <value type="value">16.2</value>
+            </key>
+            <key name="107_77">
+                <value type="value">15.9</value>
+            </key>
+            <key name="107_78">
+                <value type="value">512.1</value>
+            </key>
+            <key name="107_79">
+                <value type="value">433.4</value>
+            </key>
+            <key name="107_90">
+                <value type="value">344.4</value>
+            </key>
+            <key name="107_130">
+                <value type="value">-0.44013</value>
+            </key>
+            <key name="107_131">
+                <value type="value">43.0</value>
+            </key>
+            <key name="107_132">
+                <value type="value">41.2</value>
+            </key>
+            <key name="107_141">
+                <value type="value">12.1</value>
+            </key>
+            <key name="107_200">
+                <value type="value">91</value>
+            </key>
+            <key name="107_201">
+                <value type="value">98.8</value>
+            </key>
+            <key name="107_202">
+                <value type="value">110</value>
+            </key>
+            <key name="107_203">
+                <value type="value">85.1</value>
+            </key>
+            <key name="107_204">
+                <value type="value">85.2</value>
+            </key>
+            <key name="107_205">
+                <value type="value">410</value>
+            </key>
+            <key name="107_206">
+                <value type="value">592</value>
+            </key>
+            <key name="107_207">
+                <value type="value">1.5</value>
+            </key>
+            <key name="107_208">
+                <value type="value">44.7</value>
+            </key>
+            <key name="107_209">
+                <value type="value">11.9</value>
+            </key>
+            <key name="107_210">
+                <value type="value">221.45</value>
+            </key>
+            <key name="107_211">
+                <value type="value">13.4</value>
+            </key>
+            <key name="107_212">
+                <value type="value">0.000402</value>
+            </key>
+
+            <!-- 109- -->
+            <key name="109_3">
+                <value type="value">16.2</value>
+            </key>
+            <key name="109_7">
+                <value type="value">15.9</value>
+            </key>
+            <key name="109_8">
+                <value type="value">880</value>
+            </key>
+            <key name="109_10">
+                <value type="value">344.4</value>
+            </key>
+            <key name="109_40">
+                <value type="value">41.2</value>
+            </key>
+            <key name="109_41">
+                <value type="value">12.1</value>
+            </key>
+
+            <key name="empty">
+                <value type="value">""</value>
+            </key>
+        </key_value_mappings>
+    </databus>
+</core>

docker/conpot/dist/templates/default/template.xml

@@ -0,0 +1,78 @@
+<core>
+    <template>
+        <!-- General information about the template -->
+        <entity name="unit">S7-200</entity>
+        <entity name="vendor">Siemens</entity>
+        <entity name="description">Rough simulation of a basic Siemens S7-200 CPU with 2 slaves</entity>
+        <entity name="protocols">HTTP, MODBUS, s7comm, SNMP</entity>
+        <entity name="creator">the conpot team</entity>
+    </template>
+    <databus>
+        <!-- Core value that can be retrieved from the databus by key -->
+        <key_value_mappings>
+            <key name="FacilityName">
+                <value type="value">"DoE Water Service"</value>
+            </key>
+            <key name="SystemName">
+                <value type="value">"Central Pump"</value>
+            </key>
+            <key name="SystemDescription">
+                <value type="value">"Pump Control Unit"</value>
+            </key>
+            <key name="Uptime">
+                <value type="function">conpot.emulators.misc.uptime.Uptime</value>
+            </key>
+            <key name="sysObjectID">
+                <value type="value">"0.0"</value>
+            </key>
+            <key name="sysContact">
+                <value type="value">"DoE"</value>
+            </key>
+            <key name="sysName">
+                <value type="value">"Pump Control Unit"</value>
+            </key>
+            <key name="sysLocation">
+                <value type="value">"DoE"</value>
+            </key>
+            <key name="sysServices">
+                <value type="value">"72"</value>
+            </key>
+            <key name="memoryModbusSlave0BlockA">
+                <value type="value">[random.randint(0,1) for b in range(0,128)]</value>
+            </key>
+            <key name="memoryModbusSlave0BlockB">
+                <value type="value">[random.randint(0,1) for b in range(0,32)]</value>
+            </key>
+            <key name="memoryModbusSlave255BlockA">
+                <value type="value">[random.randint(0,1) for b in range(0,128)]</value>
+            </key>
+            <key name="memoryModbusSlave255BlockB">
+                <value type="value">[random.randint(0,1) for b in range(0,32)]</value>
+            </key>
+            <key name="memoryModbusSlave1BlockA">
+                <value type="value">[random.randint(0,1) for b in range(0,128)]</value>
+            </key>
+            <key name="memoryModbusSlave1BlockB">
+                <value type="value">[random.randint(0,1) for b in range(0,32)]</value>
+            </key>
+            <key name="memoryModbusSlave2BlockC">
+                <value type="value">[random.randint(0,1) for b in range(0,8)]</value>
+            </key>
+            <key name="memoryModbusSlave2BlockD">
+                <value type="value">[0 for b in range(0,32)]</value>
+            </key>
+            <key name="Copyright">
+                <value type="value">"Original Siemens Equipment"</value>
+            </key>
+            <key name="s7_id">
+                <value type="value">"88111222"</value>
+            </key>
+            <key name="s7_module_type">
+                <value type="value">"IM151-8 PN/DP CPU"</value>
+            </key>
+            <key name="empty">
+                <value type="value">""</value>
+            </key>
+        </key_value_mappings>
+    </databus>
+</core>

docker/conpot/dist/templates/guardian_ast/template.xml

@@ -0,0 +1,93 @@
+<core>
+    <template>
+        <!-- General information about the template -->
+        <entity name="unit">Guardian AST tank-monitoring system</entity>
+        <entity name="vendor">Guardian</entity>
+        <entity name="description">Guardian AST tank-monitoring system</entity>
+        <entity name="protocols">guardian_ast</entity>
+        <entity name="creator">the conpot team</entity>
+    </template>
+    <databus>
+        <!-- Core value that can be retrieved from the databus by key -->
+        <key_value_mappings>
+            <key name="product1">
+                <value type="value">"SUPER"</value>
+            </key>
+            <key name="product2">
+                <value type="value">"UNLEAD"</value>
+            </key>
+            <key name="product3">
+                <value type="value">"DIESEL"</value>
+            </key>
+            <key name="product4">
+                <value type="value">"ADBLUE"</value>
+            </key>
+            <key name="station_name">
+                <value type="value">"AVIA"</value>
+            </key>
+            <key name="vol1">
+                <value type="value">random.randint(1000, 9050)</value>
+            </key>
+            <key name="vol2">
+                <value type="value">random.randint(1000, 9050)</value>
+            </key>
+            <key name="vol3">
+                <value type="value">random.randint(1000, 9050)</value>
+            </key>
+            <key name="vol4">
+                <value type="value">random.randint(1000, 9050)</value>
+            </key>
+            <key name="ullage1">
+                <value type="value">random.randint(3000, 9999)</value>
+            </key>
+            <key name="ullage2">
+                <value type="value">random.randint(3000, 9999)</value>
+            </key>
+            <key name="ullage3">
+                <value type="value">random.randint(3000, 9999)</value>
+            </key>
+            <key name="ullage4">
+                <value type="value">random.randint(3000, 9999)</value>
+            </key>
+            <key name="height1">
+                <value type="value">round(random.uniform(25.00, 75.99), 2)</value>
+            </key>
+            <key name="height2">
+                <value type="value">round(random.uniform(25.00, 75.99), 2)</value>
+            </key>
+            <key name="height3">
+                <value type="value">round(random.uniform(25.00, 75.99), 2)</value>
+            </key>
+            <key name="height4">
+                <value type="value">round(random.uniform(25.00, 75.99), 2)</value>
+            </key>
+            <key name="h2o1">
+                <value type="value">round(random.uniform(0.0, 9.99), 2)</value>
+            </key>
+            <key name="h2o2">
+                <value type="value">round(random.uniform(0.0, 9.99), 2)</value>
+            </key>
+            <key name="h2o3">
+                <value type="value">round(random.uniform(0.0, 9.99), 2)</value>
+            </key>
+            <key name="h2o4">
+                <value type="value">round(random.uniform(0.0, 9.99), 2)</value>
+            </key>
+            <key name="temp1">
+                <value type="value">round(random.uniform(50.0, 59.99), 2)</value>
+            </key>
+            <key name="temp2">
+                <value type="value">round(random.uniform(50.0, 59.99), 2)</value>
+            </key>
+            <key name="temp3">
+                <value type="value">round(random.uniform(50.0, 59.99), 2)</value>
+            </key>
+            <key name="temp4">
+                <value type="value">round(random.uniform(50.0, 59.99), 2)</value>
+            </key>
+            <key name="empty">
+                <value type="value">""</value>
+            </key>
+        </key_value_mappings>
+    </databus>
+</core>

docker/conpot/dist/templates/ipmi/template.xml

@@ -0,0 +1,18 @@
+<core>
+    <template>
+        <!-- General information about the template -->
+        <entity name="unit">371</entity>
+        <entity name="vendor">IPMI</entity>
+        <entity name="description">Creates a simple IPMI device</entity>
+        <entity name="protocols">IPMI</entity>
+        <entity name="creator">Lukas Rist</entity>
+    </template>
+    <databus>
+        <!-- Core value that can be retrieved from the databus by key -->
+        <key_value_mappings>
+          <key name="SystemName">
+              <value type="value">"DoE"</value>
+          </key>
+        </key_value_mappings>
+    </databus>
+</core>

docker/conpot/dist/templates/kamstrup_382/template.xml

@@ -413,43 +413,43 @@
                 <value type="value">''</value>
             </key>
             <key name="nameserver_1">
-                <value type="value">'192.168.254.111'</value>
+                <value type="value">'0.0.0.0'</value>
             </key>
             <key name="nameserver_2">
-                <value type="value">'192.168.254.112'</value>
+                <value type="value">'0.0.0.0'</value>
             </key>
             <key name="nameserver_3">
                 <value type="value">'0.0.0.0'</value>
             </key>
             <key name="mac_address">
-                <value type="value">'00:13:EA:00:72:FA'</value>
+                <value type="value">'00:13:EA:00:00:00'</value>
             </key>
             <key name="use_dhcp">
                 <value type="value">'YES'</value>
             </key>
             <key name="ip_addr">
-                <value type="value">'192.168.201.101'</value>
+                <value type="value">'192.168.1.210'</value>
             </key>
             <key name="ip_gateway">
-                <value type="value">'192.168.201.254'</value>
+                <value type="value">'192.168.1.1'</value>
             </key>
             <key name="ip_subnet">
                 <value type="value">'255.255.255.0'</value>
             </key>
             <key name="ip_addr_dhcp">
-                <value type="value">'192.168.200.1'</value>
+                <value type="value">'192.168.0.1'</value>
             </key>
             <key name="ip_gateway_dhcp">
-                <value type="value">'192.168.200.254'</value>
+                <value type="value">'192.168.0.254'</value>
             </key>
             <key name="ip_subnet_dhcp">
                 <value type="value">'255.255.255.0'</value>
             </key>
             <key name="kap_a_server_hostname">
-                <value type="value">'de_fra_lxg00.local.dom'</value>
+                <value type="value">'pwr_ctrl_mgmt01.int.local'</value>
             </key>
             <key name="kap_a_server_ip">
-                <value type="value">'192.168.254.201'</value>
+                <value type="value">'10.232.15.242'</value>
             </key>
             <key name="kap_a_server_port">
                 <value type="value">'50'</value>

docker/conpot/dist/templates/proxy/template.xml

@@ -0,0 +1,15 @@
+    <core>
+    <template>
+        <!-- General information about the template -->
+        <entity name="unit">Proxy</entity>
+        <entity name="vendor">None</entity>
+        <entity name="description">Sample template that demonstrates the proxy feature.</entity>
+        <entity name="protocols">Proxy</entity>
+        <entity name="creator">the conpot team</entity>
+    </template>
+    <databus>
+        <!-- Core value that can be retrieved from the databus by key -->
+        <key_value_mappings>
+        </key_value_mappings>
+    </databus>
+</core>

docker/conpot/docker-compose.yml

@@ -1,19 +1,131 @@
-version: '2.1'
+# CONPOT TEMPLATE=[default, IEC104, guardian_ast, ipmi, kamstrup_382, proxy]
+version: '2.3'
 
 networks:
-  conpot_local:
+  conpot_local_default:
+  conpot_local_IEC104:
+  conpot_local_guardian_ast:
+  conpot_local_ipmi:
+  conpot_local_kamstrup_382:
 
 services:
 
-# Conpot service
-  conpot:
-    container_name: conpot
+# Conpot default service
+  conpot_default:
+    build: .
+    container_name: conpot_default
     restart: always
+    environment:
+     - CONPOT_CONFIG=/etc/conpot/conpot.cfg
+     - CONPOT_JSON_LOG=/var/log/conpot/conpot_default.json
+     - CONPOT_LOG=/var/log/conpot/conpot_default.log
+     - CONPOT_TEMPLATE=default
+     - CONPOT_TMP=/tmp/conpot
+    tmpfs:
+     - /tmp/conpot:uid=2000,gid=2000
     networks:
-     - conpot_local
+     - conpot_local_default
+    ports:
+#    - "69:69"        
+     - "80:80"
+     - "102:102"
+     - "161:161"
+     - "502:502"
+#     - "623:623"
+     - "2121:21"
+     - "44818:44818"
+     - "47808:47808"
+    image: "dtagdevsec/conpot:1811"
+    read_only: true
+    volumes:
+     - /data/conpot/log:/var/log/conpot
+
+# Conpot IEC104 service
+  conpot_IEC104:
+    build: .
+    container_name: conpot_IEC104
+    restart: always
+    environment:
+     - CONPOT_CONFIG=/etc/conpot/conpot.cfg
+     - CONPOT_JSON_LOG=/var/log/conpot/conpot_IEC104.json
+     - CONPOT_LOG=/var/log/conpot/conpot_IEC104.log
+     - CONPOT_TEMPLATE=IEC104
+     - CONPOT_TMP=/tmp/conpot
+    tmpfs:
+     - /tmp/conpot:uid=2000,gid=2000
+    networks:
+     - conpot_local_IEC104
+    ports:
+#     - "161:161"
+     - "2404:2404"
+    image: "dtagdevsec/conpot:1811"
+    read_only: true
+    volumes:
+     - /data/conpot/log:/var/log/conpot
+
+# Conpot guardian_ast service
+  conpot_guardian_ast:
+    build: .
+    container_name: conpot_guardian_ast
+    restart: always
+    environment:
+     - CONPOT_CONFIG=/etc/conpot/conpot.cfg
+     - CONPOT_JSON_LOG=/var/log/conpot/conpot_guardian_ast.json
+     - CONPOT_LOG=/var/log/conpot/conpot_guardian_ast.log
+     - CONPOT_TEMPLATE=guardian_ast
+     - CONPOT_TMP=/tmp/conpot
+    tmpfs:
+     - /tmp/conpot:uid=2000,gid=2000
+    networks:
+     - conpot_local_guardian_ast
+    ports:
+     - "10001:10001"
+    image: "dtagdevsec/conpot:1811"
+    read_only: true
+    volumes:
+     - /data/conpot/log:/var/log/conpot
+
+# Conpot ipmi
+  conpot_ipmi:
+    build: .
+    container_name: conpot_ipmi
+    restart: always
+    environment:
+     - CONPOT_CONFIG=/etc/conpot/conpot.cfg
+     - CONPOT_JSON_LOG=/var/log/conpot/conpot_ipmi.json
+     - CONPOT_LOG=/var/log/conpot/conpot_ipmi.log
+     - CONPOT_TEMPLATE=ipmi
+     - CONPOT_TMP=/tmp/conpot
+    tmpfs:
+     - /tmp/conpot:uid=2000,gid=2000
+    networks:
+     - conpot_local_ipmi
+    ports:
+     - "623:623"
+    image: "dtagdevsec/conpot:1811"
+    read_only: true
+    volumes:
+     - /data/conpot/log:/var/log/conpot
+
+# Conpot kamstrup_382
+  conpot_kamstrup_382:
+    build: .
+    container_name: conpot_kamstrup_382
+    restart: always
+    environment:
+     - CONPOT_CONFIG=/etc/conpot/conpot.cfg
+     - CONPOT_JSON_LOG=/var/log/conpot/conpot_kamstrup_382.json
+     - CONPOT_LOG=/var/log/conpot/conpot_kamstrup_382.log
+     - CONPOT_TEMPLATE=kamstrup_382
+     - CONPOT_TMP=/tmp/conpot
+    tmpfs:
+     - /tmp/conpot:uid=2000,gid=2000
+    networks:
+     - conpot_local_kamstrup_382
     ports:
      - "1025:1025"
      - "50100:50100"
-    image: "dtagdevsec/conpot:1710"
+    image: "dtagdevsec/conpot:1811"
+    read_only: true
     volumes:
      - /data/conpot/log:/var/log/conpot

docker/cowrie/Dockerfile

@@ -1,35 +1,63 @@
 FROM alpine
-MAINTAINER MO
 
 # Include dist
 ADD dist/ /root/dist/
 
 # Get and install dependencies & packages
-RUN apk -U upgrade && \
-    apk add git procps py-pip mpfr-dev openssl-dev mpc1-dev libffi-dev build-base python python-dev py-mysqldb py-requests py-setuptools gmp-dev && \
+RUN apk -U --no-cache add \
+                       build-base \
+                       git \
+                       gmp-dev \
+                       libcap \
+                       libffi-dev \
+                       mpc1-dev \
+                       mpfr-dev \
+                       openssl-dev \
+                       python \
+                       python-dev \
+                       py-mysqldb \
+                       py-pip \
+                       py-requests \
+                       py-setuptools && \
 
 # Setup user
     addgroup -g 2000 cowrie && \
-    adduser -S -s /bin/bash -u 2000 -D -g 2000 cowrie && \
+    adduser -S -s /bin/ash -u 2000 -D -g 2000 cowrie && \
 
 # Install cowrie from git
-    git clone https://github.com/micheloosterhof/cowrie.git /home/cowrie/cowrie/ && \
+    git clone --depth=1 https://github.com/micheloosterhof/cowrie /home/cowrie/cowrie/ -b v1.3.0 && \
     cd /home/cowrie/cowrie && \
-    pip install --no-cache-dir --upgrade cffi && \
-    pip install --no-cache-dir -U -r requirements.txt && \
+    pip install --no-cache-dir --upgrade cffi pip && \
+    pip install --no-cache-dir --upgrade -r requirements.txt && \
 
-# Setup user, groups and configs
+# Setup configs
+    setcap cap_net_bind_service=+ep /usr/bin/python2.7 && \
     cp /root/dist/cowrie.cfg /home/cowrie/cowrie/cowrie.cfg && \
-    cp /root/dist/userdb.txt /home/cowrie/cowrie/data/userdb.txt && \
-    chown cowrie:cowrie -R /home/cowrie/* && \
+    chown cowrie:cowrie -R /home/cowrie/* /usr/lib/python2.7/site-packages/twisted/plugins && \
+
+# Start Cowrie once to prevent dropin.cache errors upon container start caused by read-only filesystem
+    su - cowrie -c "export PYTHONPATH=/home/cowrie/cowrie && \
+                   cd /home/cowrie/cowrie && \
+                   /usr/bin/twistd --uid=2000 --gid=2000 -y cowrie.tac --pidfile cowrie.pid cowrie &" && \
+    sleep 10 && \
 
 # Clean up
+    apk del --purge build-base \
+                    git \
+                    gmp-dev \
+                    libcap \
+                    libffi-dev \
+                    mpc1-dev \
+                    mpfr-dev \
+                    python-dev \
+                    py-mysqldb \
+                    py-pip && \
     rm -rf /root/* && \
-    apk del git py-pip mpfr-dev mpc1-dev libffi-dev build-base py-mysqldb gmp-dev python-dev && \
-    rm -rf /var/cache/apk/*
+    rm -rf /var/cache/apk/* && \
+    rm -rf /home/cowrie/cowrie/cowrie.pid
 
 # Start cowrie
 ENV PYTHONPATH /home/cowrie/cowrie
 WORKDIR /home/cowrie/cowrie
-USER cowrie
-CMD ["/usr/bin/twistd", "--nodaemon", "-y", "cowrie.tac", "--pidfile", "var/run/cowrie.pid", "cowrie"]
+USER cowrie:cowrie
+CMD ["/usr/bin/twistd", "--nodaemon", "-y", "cowrie.tac", "--pidfile", "/tmp/cowrie/cowrie.pid", "cowrie"]

docker/cowrie/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/cowrie:1710.svg)](https://microbadger.com/images/dtagdevsec/cowrie:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/cowrie:1710.svg)](https://microbadger.com/images/dtagdevsec/cowrie:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/cowrie:1811.svg)](https://microbadger.com/images/dtagdevsec/cowrie:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/cowrie:1811.svg)](https://microbadger.com/images/dtagdevsec/cowrie:1811 "Get your own image badge on microbadger.com")
 
 # cowrie
 
@@ -12,4 +12,4 @@ The `docker-compose.yml` contains the necessary settings to test cowrie using `d
 
 # Cowrie Dashboard
 
-![Cowrie Dashboard](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/docker/cowrie/doc/dashboard.png)
+![Cowrie Dashboard](doc/dashboard.png)

docker/cowrie/dist/cowrie.cfg

@@ -1,472 +1,25 @@
-#
-# Cowrie configuration file (cowrie.cfg)
-#
-
-# ============================================================================
-# General Honeypot Options
-# ============================================================================
 [honeypot]
-
-# Sensor name is used to identify this Cowrie instance. Used by the database
-# logging modules such as mysql.
-#
-# If not specified, the logging modules will instead use the IP address of the
-# server as the sensor name.
-#
-# (default: not specified)
-#sensor_name=t-pot
-
-# Hostname for the honeypot. Displayed by the shell prompt of the virtual
-# environment
-#
-# (default: svr04)
 hostname = ubuntu
-
-
-# Directory where to save log files in.
-#
-# (default: log)
-log_path = log
-
-
-# Directory where to save downloaded artifacts in.
-#
-# (default: dl)
-download_path = dl
-
-
-# Directory for miscellaneous data files, such as the password database.
-#
-# (default: data_path)
-data_path = data
-
-
-# Directory where virtual file contents are kept in.
-#
-# This is only used by commands like 'cat' to display the contents of files.
-# Adding files here is not enough for them to appear in the honeypot - the
-# actual virtual filesystem is kept in filesystem_file (see below)
-#
-# (default: honeyfs)
-contents_path = honeyfs
-
-
-# File in the Python pickle format containing the virtual filesystem.
-#
-# This includes the filenames, paths, permissions for the Cowrie filesystem,
-# but not the file contents. This is created by the bin/createfs utility from
-# a real template linux installation.
-#
-# (default: fs.pickle)
-filesystem_file = data/fs.pickle
-
-
-# Directory for creating simple commands that only output text.
-#
-# The command must be placed under this directory with the proper path, such
-# as:
-#   txtcmds/usr/bin/vi
-# The contents of the file will be the output of the command when run inside
-# the honeypot.
-#
-# In addition to this, the file must exist in the virtual filesystem
-#
-# (default: txtcmds)
-txtcmds_path = txtcmds
-
-
-# Maximum file size (in bytes) for downloaded files to be stored in 'download_path'.
-# A value of 0 means no limit. If the file size is known to be too big from the start,
-# the file will not be stored on disk at all.
-#
-# (default: 0)
-#download_limit_size = 10485760
-
-
-# TTY logging will log a transcript of the complete terminal interaction in UML
-# compatible format.
-# (default: true)
-ttylog = true
-
-
-
-# ============================================================================
-# Network Specific Options
-# ============================================================================
-
-
-# IP address to bind to when opening outgoing connections. Used by wget and
-# curl commands.
-#
-# (default: not specified)
-#out_addr = 0.0.0.0
-
-
-# Fake address displayed as the address of the incoming connection.
-# This doesn't affect logging, and is only used by honeypot commands such as
-# 'w' and 'last'
-#
-# If not specified, the actual IP address is displayed instead (default
-# behaviour).
-#
-# (default: not specified)
-#fake_addr = 192.168.66.254
-
-
-# The IP address on which this machine is reachable on from the internet.
-# Useful if you use portforwarding or other mechanisms. If empty, Cowrie
-# will determine by itself. Used in 'netstat' output
-#
-#internet_facing_ip = 9.9.9.9
-
-
-# Enable to log the public IP of the honeypot (useful if listening on 127.0.0.1)
-# IP address is obtained by querying http://myip.threatstream.com
 report_public_ip = true
-
-
-
-# ============================================================================
-# Authentication Specific Options
-# ============================================================================
-
-
-# Class that implements the checklogin() method.
-#
-# Class must be defined in cowrie/core/auth.py
-# Default is the 'UserDB' class which uses the password database.
-#
-# Alternatively the 'AuthRandom' class can be used, which will let
-# a user login after a random number of attempts.
-# It will also cache username/password combinations that allow login.
-#
-#auth_class = UserDB
-
-# When AuthRandom is used also set the
-#  auth_class_parameters: <min try>, <max try>, <maxcache>
-#  for example: 2, 5, 10 = allows access after randint(2,5) attempts
-#  and cache 10 combinations.
-#
 auth_class = AuthRandom
 auth_class_parameters = 2, 5, 10
-
-
-# No authentication checking at all
-# enabling 'auth_none' will enable the ssh2 'auth_none' authentication method
-# this allows the requested user in without any verification at all
-#
-# (default: false)
-#auth_none_enabled = false
-
-
-
-# ============================================================================
-# Historical SSH Specific Options
-# historical options in [honeypot] that have not yet been moved to [ssh]
-# ============================================================================
-
-# Source Port to report in logs (useful if you use iptables to forward ports to Cowrie)
 reported_ssh_port = 22
+data_path = /tmp/cowrie/data
 
-
-
-# ============================================================================
-# SSH Specific Options
-# ============================================================================
 [ssh]
-
-# Enable SSH support
-# (default: true)
-enabled = true
-
-
-# Public and private SSH key files. If these don't exist, they are created
-# automatically.
-rsa_public_key = etc/ssh_host_rsa_key.pub
-rsa_private_key = etc/ssh_host_rsa_key
-dsa_public_key = etc/ssh_host_dsa_key.pub
-dsa_private_key = etc/ssh_host_dsa_key
-
-# SSH Version String
-#
-# Use these to disguise your honeypot from a simple SSH version scan
-# Examples:
-# SSH-2.0-OpenSSH_5.1p1 Debian-5
-# SSH-1.99-OpenSSH_4.3
-# SSH-1.99-OpenSSH_4.7
-# SSH-1.99-Sun_SSH_1.1
-# SSH-2.0-OpenSSH_4.2p1 Debian-7ubuntu3.1
-# SSH-2.0-OpenSSH_4.3
-# SSH-2.0-OpenSSH_4.6
-# SSH-2.0-OpenSSH_5.1p1 Debian-5
-# SSH-2.0-OpenSSH_5.1p1 FreeBSD-20080901
-# SSH-2.0-OpenSSH_5.3p1 Debian-3ubuntu5
-# SSH-2.0-OpenSSH_5.3p1 Debian-3ubuntu6
-# SSH-2.0-OpenSSH_5.3p1 Debian-3ubuntu7
-# SSH-2.0-OpenSSH_5.5p1 Debian-6
-# SSH-2.0-OpenSSH_5.5p1 Debian-6+squeeze1
-# SSH-2.0-OpenSSH_5.5p1 Debian-6+squeeze2
-# SSH-2.0-OpenSSH_5.8p2_hpn13v11 FreeBSD-20110503
-# SSH-2.0-OpenSSH_5.9p1 Debian-5ubuntu1
-# SSH-2.0-OpenSSH_6.0p1 Debian-4+deb7u2
-# SSH-2.0-OpenSSH_5.9
-#
-# (default: "SSH-2.0-SSH-2.0-OpenSSH_6.0p1 Debian-4+deb7u2")
 version = SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.2
+listen_endpoints = tcp:22:interface=0.0.0.0
 
-
-# IP addresses to listen for incoming SSH connections.
-#
-# (default: 0.0.0.0) = any IPv4 address
-#listen_addr = 0.0.0.0
-# (use :: for listen to all IPv6 and IPv4 addresses)
-#listen_addr = ::
-
-
-# Port to listen for incoming SSH connections.
-#
-# (default: 2222)
-#listen_port = 22
-
-
-# sftp_enabled enables the sftp subsystem
-sftp_enabled = true
-
-
-# Enable SSH direct-tcpip forwarding
-# (default: true)
-forwarding = true
-
-
-# This enables redirecting forwarding requests to another address
-# Useful for forwarding protocols to other honeypots
-# (default: false)
-forward_redirect = false
-
-
-# Configure where to forward the data to.
-# forward_redirect_<portnumber> = <redirect ip>:<redirect port>
-
-# Redirect http/https
-forward_redirect_80 = 127.0.0.1:8000
-forward_redirect_443 = 127.0.0.1:8443
-
-# To record SMTP traffic, install an SMTP honeypoint.
-# (e.g https://github.com/awhitehatter/mailoney), run
-# python mailoney.py -s yahoo.com -t schizo_open_relay -p 12525
-forward_redirect_25 = 127.0.0.1:12525
-forward_redirect_587 = 127.0.0.1:12525
-
-
-
-# ============================================================================
-# Telnet Specific Options
-# ============================================================================
 [telnet]
-
-# Enable Telnet support, disabled by default
 enabled = true
-
-# IP addresses to listen for incoming Telnet connections.
-#
-# (default: 0.0.0.0) = any IPv4 address
-#listen_addr = 0.0.0.0
-# (use :: for listen to all IPv6 and IPv4 addresses)
-#listen_addr = ::
-
-
-# Port to listen for incoming Telnet connections.
-#
-# (default: 2223)
-#listen_port = 23
-
-# Source Port to report in logs (useful if you use iptables to forward ports to Cowrie)
+listen_endpoints = tcp:23:interface=0.0.0.0
 reported_port = 23
 
-
-
-# ============================================================================
-# Database logging Specific Options
-# ============================================================================
-
-# XMPP Logging
-# Log to an xmpp server.
-#
-#[database_xmpp]
-#server = sensors.carnivore.it
-#user = anonymous@sensors.carnivore.it
-#password = anonymous
-#muc = dionaea.sensors.carnivore.it
-#signal_createsession = cowrie-events
-#signal_connectionlost = cowrie-events
-#signal_loginfailed = cowrie-events
-#signal_loginsucceeded = cowrie-events
-#signal_command = cowrie-events
-#signal_clientversion = cowrie-events
-#debug=true
-
-
-
-# ============================================================================
-# Output Plugins
-# These provide an extensible mechanism to send audit log entries to third
-# parties. The audit entries contain information on clients connecting to
-# the honeypot.
-# ============================================================================
-
-
-# JSON based logging module
-#
 [output_jsonlog]
+enabled = true
 logfile = log/cowrie.json
 
-
-# Supports logging to Elasticsearch
-# This is a simple early release
-#
-#[output_elasticsearch]
-#host = localhost
-#port = 9200
-#index = cowrie
-#type = cowrie
-
-
-# Send login attemp information to SANS DShield
-# See https://isc.sans.edu/ssh.html
-# You must signup for an api key.
-# Once registered, find your details at: https://isc.sans.edu/myaccount.html
-#
-#[output_dshield]
-#userid = userid_here
-#auth_key = auth_key_here
-#batch_size = 100
-
-
-# Local Syslog output module
-#
-# This sends log messages to the local syslog daemon.
-# Facility can be:
-# KERN, USER, MAIL, DAEMON, AUTH, LPR, NEWS, UUCP, CRON, SYSLOG and LOCAL0 to LOCAL7.
-#
-# Format can be:
-# text, cef
-#
-#[output_localsyslog]
-#facility = USER
-#format = text
-
-
-# Text output
-# This writes audit log entries to a text file
-#
-# Format can be:
-# text, cef
-#
 [output_textlog]
+enabled = false
 logfile = log/cowrie-textlog.log
 format = text
-
-
-# MySQL logging module
-# Database structure for this module is supplied in doc/sql/mysql.sql
-#
-# MySQL logging requires extra software: sudo apt-get install libmysqlclient-dev
-# MySQL logging requires an extra Python module: pip install mysql-python
-#
-#[output_mysql]
-#host = localhost
-#database = cowrie
-#username = cowrie
-#password = secret
-#port = 3306
-#debug = false
-
-# Rethinkdb output module
-# Rethinkdb output module requires extra Python module: pip install rethinkdb
-
-#[output_rethinkdblog]
-#host = 127.0.0.1
-#port = 28015
-#table = output
-#password =
-#db = cowrie
-
-# SQLite3 logging module
-#
-# Logging to SQLite3 database. To init the database, use the script
-# doc/sql/sqlite3.sql:
-#     sqlite3 <db_file> < doc/sql/sqlite3.sql
-#
-#[output_sqlite]
-#db_file = cowrie.db
-
-# MongoDB logging module
-#
-# MongoDB logging requires an extra Python module: pip install pymongo
-#
-#[output_mongodb]
-#connection_string = mongodb://username:password@host:port/database
-#database = dbname
-
-
-# Splunk SDK output module - Legacy. Requires Splunk API installed
-# This sends logs directly to Splunk using the Python REST SDK
-#
-#[output_splunklegacy]
-#host = localhost
-#port = 8889
-#username = admin
-#password = password
-#index = cowrie
-
-
-# Splunk HTTP Event Collector (HEC) output module
-# Sends JSON directly to Splunk over HTTPS
-# mandatory fields: url, token
-# optional fields: index, source, sourcetype, host
-#
-#[output_splunk]
-#url = https://localhost:8088/services/collector/event
-#token = 6A0EA6C6-8006-4E39-FC44-C35FF6E561A8
-#index = cowrie
-#sourcetype = cowrie
-#source = cowrie
-
-
-# HPFeeds
-#
-#[output_hpfeeds]
-#server = hpfeeds.mysite.org
-#port = 10000
-#identifier = abc123
-#secret = secret
-#debug=false
-
-
-# VirusTotal output module
-# You must signup for an api key.
-#
-#[output_virustotal]
-#api_key = 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
-#
-
-#[output_slack]
-# This will produce a _lot_ of messages - you have been warned....
-#channel = channel_that_events_should_be_posted_in
-#token = slack_token_for_your_bot
-##debug=true
-
-
-# https://csirtg.io
-# You must signup for an api key.
-#
-#[output_csirtg]
-#username=wes
-#feed=scanners
-#description=random scanning activity
-#token=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
-
-
-#[output_socketlog]
-#address = 127.0.0.1:9000
-#timeout = 5

docker/cowrie/dist/userdb.txt

@@ -1 +0,0 @@
-root:0:password

docker/cowrie/docker-compose.yml

@@ -1,6 +1,4 @@
-# T-Pot (Standard)
-# For docker-compose ...
-version: '2.1'
+version: '2.3'
 
 networks:
   cowrie_local:
@@ -9,16 +7,19 @@ services:
 
 # Cowrie service
   cowrie:
+    build: .
     container_name: cowrie
     restart: always
+    tmpfs:
+     - /tmp/cowrie:uid=2000,gid=2000
+     - /tmp/cowrie/data:uid=2000,gid=2000
     networks:
      - cowrie_local
-    cap_add:
-     - NET_BIND_SERVICE
     ports:
-     - "22:2222"
-     - "23:2223"
-    image: "dtagdevsec/cowrie:1710"
+     - "22:22"
+     - "23:23"
+    image: "dtagdevsec/cowrie:1811"
+    read_only: true
     volumes:
      - /data/cowrie/downloads:/home/cowrie/cowrie/dl
      - /data/cowrie/keys:/home/cowrie/cowrie/etc

docker/cyberchef/Dockerfile

@@ -0,0 +1,36 @@
+FROM alpine
+
+# Get and install dependencies & packages
+RUN apk -U --no-cache add \
+            curl \
+            git \
+            npm \
+            nodejs && \
+    npm install -g grunt-cli && \
+    npm install -g http-server && \
+    npm install npm@latest -g && \
+
+# Install CyberChef 
+    cd /root && \
+    git clone https://github.com/gchq/cyberchef -b v7.11.1 --depth=1 && \
+    chown -R nobody:nobody cyberchef && \
+    cd cyberchef && \
+    npm install && \
+    grunt prod && \
+    mkdir -p /opt/cyberchef && \
+    mv build/prod/* /opt/cyberchef && \
+    cd / && \
+
+# Clean up
+    apk del --purge git \
+                    npm && \
+    rm -rf /root/* && \
+    rm -rf /var/cache/apk/*
+
+# Healthcheck
+HEALTHCHECK --retries=10 CMD curl -s -XGET 'http://127.0.0.1:8000'
+
+# Set user, workdir and start spiderfoot
+USER nobody:nobody
+WORKDIR /opt/cyberchef
+CMD ["http-server", "-p", "8000"]

docker/cyberchef/README.md

@@ -0,0 +1,15 @@
+[![](https://images.microbadger.com/badges/version/dtagdevsec/cyberchef:1811.svg)](https://microbadger.com/images/dtagdevsec/cyberchef:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/cyberchef:1811.svg)](https://microbadger.com/images/dtagdevsec/cyberchef:1811 "Get your own image badge on microbadger.com")
+
+# cyberchef
+
+[cyberchef](https://github.com/gchq/CyberChef) The Cyber Swiss Army Knife - a web app for encryption, encoding, compression and data analysis.
+
+This dockerized version is part of the **[T-Pot community honeypot](http://dtag-dev-sec.github.io/)** of Deutsche Telekom AG.
+
+The `Dockerfile` contains the blueprint for the dockerized version and will be used to setup the docker image.
+
+The `docker-compose.yml` contains the necessary settings to test cyberchef using `docker-compose`. This will ensure to start the docker container with the appropriate permissions and port mappings.
+
+# CyberChef UI
+
+![Cyberchef](doc/dashboard.png)

docker/cyberchef/docker-compose.yml

@@ -0,0 +1,18 @@
+version: '2.3'
+
+networks:
+  cyberchef_local:
+
+services:
+
+# Cyberchef service
+  cyberchef:
+    build: .
+    container_name: cyberchef
+    restart: always
+    networks:
+     - cyberchef_local
+    ports:
+     - "127.0.0.1:64299:8000"
+    image: "dtagdevsec/cyberchef:1811"
+    read_only: true

docker/dionaea/Dockerfile

@@ -1,5 +1,4 @@
 FROM debian:stretch-slim
-MAINTAINER MO
 ENV DEBIAN_FRONTEND noninteractive
 
 # Include dist
@@ -7,55 +6,46 @@ ADD dist/ /root/dist/
 
 # Install dependencies and packages
 RUN apt-get update -y && \
-    apt-get upgrade -y && \
+    apt-get dist-upgrade -y && \
     apt-get install -y --no-install-recommends \
-      autoconf \
-      automake \
-      build-essential \
-      ca-certificates \
-      check \
-      cython3 \
-      git \
-      libcurl4-openssl-dev \
-      libemu-dev \
-      libev-dev \
-      libglib2.0-dev \
-      libloudmouth1-dev \
-      libnetfilter-queue-dev \
-      libnl-3-dev \
-      libpcap-dev \
-      libssl-dev \
-      libtool \
-      libudns-dev \
-      procps \
-      python3 \
-      python3-dev \
-      python3-bson \
-      python3-yaml && \
+	build-essential \
+	ca-certificates \
+	check \
+	cmake \
+	cython3 \
+	git \
+        libcap2-bin \
+	libcurl4-openssl-dev \
+	libemu-dev \
+	libev-dev \
+	libglib2.0-dev \
+	libloudmouth1-dev \
+	libnetfilter-queue-dev \
+	libnl-3-dev \
+	libpcap-dev \
+	libssl-dev \
+	libtool \
+	libudns-dev \
+	procps \
+	python3 \
+	python3-dev \
+	python3-bson \
+	python3-yaml \
+	ttf-liberation && \
 
 # Get and install dionaea
-    git clone https://github.com/dinotools/dionaea /root/dionaea/ && \
+    git clone --depth=1 https://github.com/dinotools/dionaea -b 0.8.0 /root/dionaea/ && \
     cd /root/dionaea && \
-#    git checkout 99e9cfc88cfa8f3715813b18ec7006bca2622d76 && \
-    autoreconf -vi && \
-    ./configure \
-      --prefix=/opt/dionaea \
-      --with-python=/usr/bin/python3 \
-      --with-cython-dir=/usr/bin \
-      --enable-ev \
-      --with-ev-include=/usr/include \
-      --with-ev-lib=/usr/lib \
-      --with-emu-lib=/usr/lib/libemu \
-      --with-emu-include=/usr/include \
-      --with-nl-include=/usr/include/libnl3 \
-      --with-nl-lib=/usr/lib \
-      --enable-static && \
+    mkdir build && \
+    cd build && \
+    cmake -DCMAKE_INSTALL_PREFIX:PATH=/opt/dionaea .. && \
     make && \
     make install && \
 
 # Setup user and groups
     addgroup --gid 2000 dionaea && \
     adduser --system --no-create-home --shell /bin/bash --uid 2000 --disabled-password --disabled-login --gid 2000 dionaea && \
+    setcap cap_net_bind_service=+ep /opt/dionaea/bin/dionaea && \
 
 # Supply configs and set permissions
     chown -R dionaea:dionaea /opt/dionaea/var && \
@@ -64,11 +54,10 @@ RUN apt-get update -y && \
 
 # Setup runtime and clean up
     apt-get purge -y \
-      autoconf \
-      automake \
       build-essential \
       ca-certificates \
       check \
+      cmake \
       cython3 \
       git \
       libcurl4-openssl-dev \
@@ -85,7 +74,7 @@ RUN apt-get update -y && \
       python3 \
       python3-dev \   
       python3-bson \
-      python3-yaml && \
+      python3-yaml && \ 
 
     apt-get install -y \
       ca-certificates \
@@ -107,4 +96,5 @@ RUN apt-get update -y && \
     rm -rf /root/* /var/lib/apt/lists/* /tmp/* /var/tmp/*
 
 # Start dionaea
+USER dionaea:dionaea
 CMD ["/opt/dionaea/bin/dionaea", "-u", "dionaea", "-g", "dionaea", "-c", "/opt/dionaea/etc/dionaea/dionaea.cfg"]

docker/dionaea/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/dionaea:1710.svg)](https://microbadger.com/images/dtagdevsec/dionaea:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/dionaea:1710.svg)](https://microbadger.com/images/dtagdevsec/dionaea:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/dionaea:1811.svg)](https://microbadger.com/images/dtagdevsec/dionaea:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/dionaea:1811.svg)](https://microbadger.com/images/dtagdevsec/dionaea:1811 "Get your own image badge on microbadger.com")
 
 # dionaea
 
@@ -12,4 +12,4 @@ The `docker-compose.yml` contains the necessary settings to test dionaea using `
 
 # Dionaea Dashboard
 
-![Dionaea Dashboard](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/docker/dionaea/doc/dashboard.png)
+![Dionaea Dashboard](doc/dashboard.png)

docker/dionaea/dist/etc/dionaea.cfg

@@ -8,11 +8,11 @@ listen.mode=getifaddrs
 # listen.interfaces=eth0,tap0
 
 # Country
-ssl.default.c=CN
+ssl.default.c=AU
 # Common Name/domain name
-ssl.default.cn=
+ssl.default.cn=Some-State
 # Organization
-ssl.default.o=
+ssl.default.o=Internet Widgits Pty Ltd
 # Organizational Unit
 ssl.default.ou=
 

docker/dionaea/dist/etc/ihandlers/emu_scripts.yaml

@@ -0,0 +1,9 @@
+- name: emu_scripts
+  config:
+    # Maximum dumber of subdownloads
+    # max_subdownloads: 20
+    enabled_handlers:
+      - powershell
+      - vbscript
+      # Be very careful if you use the raw_url handler
+      #- raw_url

docker/dionaea/dist/etc/services/http.yaml

@@ -2,7 +2,7 @@
   config:
     root: "/opt/dionaea/var/dionaea/roots/www"
     ports:
-      - 80
+      - 81
     ssl_ports:
       - 443
     max_request_size: 32768 # maximum size in kbytes of the request (32MB)

docker/dionaea/docker-compose.yml

@@ -1,27 +1,21 @@
-# T-Pot (Standard)
-# For docker-compose ...
-version: '2.1'
-
-networks:
-  dionaea_local:
+version: '2.3'
 
 services:
 
 # Dionaea service
   dionaea:
+    build: .
     container_name: dionaea
     stdin_open: true
+    tty: true
     restart: always
-    networks:
-     - dionaea_local
-    cap_add:
-     - NET_BIND_SERVICE
+    network_mode: "host"
     ports:
      - "20:20"
      - "21:21"
      - "42:42"
      - "69:69/udp"
-     - "8081:80"
+     - "81:81"
      - "135:135"
      - "443:443"
      - "445:445"
@@ -33,7 +27,8 @@ services:
      - "5060:5060/udp"
      - "5061:5061"
      - "27017:27017"
-    image: "dtagdevsec/dionaea:1710"
+    image: "dtagdevsec/dionaea:1811"
+    read_only: true
     volumes:
      - /data/dionaea/roots/ftp:/opt/dionaea/var/dionaea/roots/ftp
      - /data/dionaea/roots/tftp:/opt/dionaea/var/dionaea/roots/tftp

docker/elasticpot/Dockerfile

@@ -1,30 +1,34 @@
 FROM alpine
-MAINTAINER MS/MO
 
 # Include dist
 ADD dist/ /root/dist/
 
 # Install packages
-RUN apk -U upgrade && \ 
-    apk add bash python3 git && \
-    pip3 install --upgrade pip && \
-    pip3 install bottle requests configparser datetime && \
+RUN apk -U --no-cache add \
+             git \
+             python3 && \
+    pip3 install --no-cache-dir --upgrade pip && \
+    pip3 install --no-cache-dir bottle \
+                                configparser \
+                                datetime \
+                                requests && \
     mkdir -p /opt && \
     cd /opt/ && \
-    git clone https://github.com/schmalle/ElasticpotPY.git && \
+    git clone --depth=1 https://github.com/schmalle/ElasticpotPY.git && \
 
 # Setup user, groups and configs
     addgroup -g 2000 elasticpot && \
-    adduser -S -H -s /bin/bash -u 2000 -D -g 2000 elasticpot && \
+    adduser -S -H -s /bin/ash -u 2000 -D -g 2000 elasticpot && \
     mv /root/dist/elasticpot.cfg /opt/ElasticpotPY/ && \
     mkdir /opt/ElasticpotPY/log && \
 
 # Clean up
-    apk del git && \
+    apk del --purge git && \
     rm -rf /root/* && \
     rm -rf /var/cache/apk/*
 
 # Start elasticpot
-USER elasticpot
+STOPSIGNAL SIGINT
+USER elasticpot:elasticpot
 WORKDIR /opt/ElasticpotPY/
 CMD ["/usr/bin/python3","main.py"]

docker/elasticpot/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/elasticpot:1710.svg)](https://microbadger.com/images/dtagdevsec/elasticpot:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/elasticpot:1710.svg)](https://microbadger.com/images/dtagdevsec/elasticpot:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/elasticpot:1811.svg)](https://microbadger.com/images/dtagdevsec/elasticpot:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/elasticpot:1811.svg)](https://microbadger.com/images/dtagdevsec/elasticpot:1811 "Get your own image badge on microbadger.com")
 
 # elasticpot
 
@@ -12,4 +12,4 @@ The `docker-compose.yml` contains the necessary settings to test elasticpot usin
 
 # ElasticPot Dashboard
 
-![ElasticPot Dashboard](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/docker/elasticpot/doc/dashboard.png)
+![ElasticPot Dashboard](doc/dashboard.png)

docker/elasticpot/docker-compose.yml

@@ -1,4 +1,4 @@
-version: '2.1'
+version: '2.3'
 
 networks:
   elasticpot_local:
@@ -7,12 +7,14 @@ services:
 
 # Elasticpot service
   elasticpot:
+    build: .
     container_name: elasticpot
     restart: always
     networks:
      - elasticpot_local
     ports:
      - "9200:9200"
-    image: "dtagdevsec/elasticpot:1710"
+    image: "dtagdevsec/elasticpot:1811"
+    read_only: true
     volumes:
      - /data/elasticpot/log:/opt/ElasticpotPY/log

docker/elk/README.md

@@ -1,11 +1,11 @@
 # Elasticsearch
-[![](https://images.microbadger.com/badges/version/dtagdevsec/elasticsearch:1710.svg)](https://microbadger.com/images/dtagdevsec/elasticsearch:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/elasticsearch:1710.svg)](https://microbadger.com/images/dtagdevsec/elasticsearch:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/elasticsearch:1811.svg)](https://microbadger.com/images/dtagdevsec/elasticsearch:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/elasticsearch:1811.svg)](https://microbadger.com/images/dtagdevsec/elasticsearch:1811 "Get your own image badge on microbadger.com")
 
 # Logstash
-[![](https://images.microbadger.com/badges/version/dtagdevsec/logstash:1710.svg)](https://microbadger.com/images/dtagdevsec/logstash:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/logstash:1710.svg)](https://microbadger.com/images/dtagdevsec/logstash:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/logstash:1811.svg)](https://microbadger.com/images/dtagdevsec/logstash:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/logstash:1811.svg)](https://microbadger.com/images/dtagdevsec/logstash:1811 "Get your own image badge on microbadger.com")
 
 # Kibana
-[![](https://images.microbadger.com/badges/version/dtagdevsec/kibana:1710.svg)](https://microbadger.com/images/dtagdevsec/kibana:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/kibana:1710.svg)](https://microbadger.com/images/dtagdevsec/kibana:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/kibana:1811.svg)](https://microbadger.com/images/dtagdevsec/kibana:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/kibana:1811.svg)](https://microbadger.com/images/dtagdevsec/kibana:1811 "Get your own image badge on microbadger.com")
 
 # elk stack
 
@@ -17,4 +17,4 @@ The `Dockerfiles` contain the blueprint for the dockerized elk stack and will be
 
 # T-Pot Dashboard
 
-![T-Pot Dashboard](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/docker/elk/doc/dashboard.png)
+![T-Pot Dashboard](doc/dashboard.png)

docker/elk/docker-compose.yml

@@ -0,0 +1,70 @@
+version: '2.3'
+
+services:
+
+# ELK services
+## Elasticsearch service
+  elasticsearch:
+    build: elasticsearch/.
+    container_name: elasticsearch
+    restart: always
+    environment:
+     - bootstrap.memory_lock=true
+     - ES_JAVA_OPTS=-Xms1024m -Xmx1024m
+     - ES_TMPDIR=/tmp
+    cap_add:
+     - IPC_LOCK
+    ulimits:
+      memlock:
+        soft: -1
+        hard: -1
+      nofile:
+        soft: 65536
+        hard: 65536
+    mem_limit: 4g
+    ports:
+     - "127.0.0.1:64298:9200"
+    image: "dtagdevsec/elasticsearch:1811"
+    volumes:
+     - /data:/data
+
+## Kibana service
+  kibana:
+    build: kibana/.
+    container_name: kibana
+    restart: always
+    stop_signal: SIGKILL
+    depends_on:
+      elasticsearch:
+        condition: service_healthy
+    ports:
+     - "127.0.0.1:64296:5601"
+    image: "dtagdevsec/kibana:1811"
+
+## Logstash service
+  logstash:
+    build: logstash/.
+    container_name: logstash
+    restart: always
+    depends_on:
+      elasticsearch:
+        condition: service_healthy
+    env_file:
+     - /opt/tpot/etc/compose/elk_environment
+    image: "dtagdevsec/logstash:1811"
+    volumes:
+     - /data:/data
+     - /root/tpotce/docker/elk/logstash/dist/logstash.conf:/etc/logstash/conf.d/logstash.conf
+
+## Elasticsearch-head service
+  head:
+    build: head/.
+    container_name: head
+    restart: always
+    depends_on:
+      elasticsearch:
+        condition: service_healthy
+    ports:
+     - "127.0.0.1:64302:9100"
+    image: "dtagdevsec/head:1811"
+    read_only: true

docker/elk/elasticsearch/Dockerfile

@@ -1,18 +1,20 @@
 FROM alpine
-MAINTAINER MO
 
 # Include dist
 ADD dist/ /root/dist/
 
 # Setup env and apt
-RUN apk -U upgrade && \
-    apk add bash curl openjdk8-jre procps wget && \
+RUN apk -U add \
+             bash \
+             curl \
+             openjdk8-jre \
+             wget && \
 
 # Get and install packages
     cd /root/dist/ && \
     mkdir -p /usr/share/elasticsearch/ && \
-    wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.9.tar.gz && \
-    tar xvfz elasticsearch-5.6.9.tar.gz --strip-components=1 -C /usr/share/elasticsearch/ && \
+    wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.5.1.tar.gz && \
+    tar xvfz elasticsearch-6.5.1.tar.gz --strip-components=1 -C /usr/share/elasticsearch/ && \
 
 # Add and move files
     cd /root/dist/ && \
@@ -21,16 +23,19 @@ RUN apk -U upgrade && \
 
 # Setup user, groups and configs
     addgroup -g 2000 elasticsearch && \
-    adduser -S -H -s /bin/bash -u 2000 -D -g 2000 elasticsearch && \
+    adduser -S -H -s /bin/ash -u 2000 -D -g 2000 elasticsearch && \
     chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/ && \
+    rm -rf /usr/share/elasticsearch/modules/x-pack-ml && \
 
 # Clean up
-    apk del wget && \
-    rm -rf /root/*
+    apk del --purge wget && \
+    rm -rf /root/* && \
+    rm -rf /tmp/* && \
+    rm -rf /var/cache/apk/*
 
 # Healthcheck
 HEALTHCHECK --retries=10 CMD curl -s -XGET 'http://127.0.0.1:9200/_cat/health'
 
 # Start ELK
-USER elasticsearch
+USER elasticsearch:elasticsearch
 CMD ["/usr/share/elasticsearch/bin/elasticsearch"]

docker/elk/elasticsearch/dist/elasticsearch.yml

@@ -1,5 +1,6 @@
 cluster.name: tpotcluster
 node.name: "tpotcluster-node-01"
+xpack.ml.enabled: false
 path:
     logs: /data/elk/log
     data: /data/elk/data

docker/elk/elasticsearch/docker-compose.yml

@@ -0,0 +1,29 @@
+version: '2.3'
+
+services:
+
+# ELK services
+## Elasticsearch service
+  elasticsearch:
+    build: .
+    container_name: elasticsearch
+    restart: always
+    environment:
+     - bootstrap.memory_lock=true
+     - ES_JAVA_OPTS=-Xms1024m -Xmx1024m
+     - ES_TMPDIR=/tmp
+    cap_add:
+     - IPC_LOCK
+    ulimits:
+      memlock:
+        soft: -1
+        hard: -1
+      nofile:
+        soft: 65536
+        hard: 65536
+    mem_limit: 2g
+    ports:
+     - "127.0.0.1:64298:9200"
+    image: "dtagdevsec/elasticsearch:1811"
+    volumes:
+     - /data:/data

docker/elk/head/Dockerfile

@@ -1,32 +1,34 @@
-# Elasticsearch-head Dockerfile by MO
-#
-# VERSION 17.06 
 FROM alpine
-MAINTAINER MO
 
 # Setup env and apt
-RUN apk -U upgrade && \
-    apk add bash curl nodejs nodejs-npm git procps && \
+RUN apk -U add \
+            curl \
+            git \
+            nodejs \
+            nodejs-npm && \
 
 # Get and install packages
     mkdir -p /usr/src/app/ && \
     cd /usr/src/app/ && \
-    git clone https://github.com/mobz/elasticsearch-head . && \
+    git clone --depth=1 https://github.com/mobz/elasticsearch-head . && \
     npm install http-server && \
-    sed -i 's/\"http\:\/\/localhost\:9200\"/\"https\:\/\/\<FQDN\>\:64297\/es\/\"/' /usr/src/app/_site/app.js && \
+    sed -i "s#\"http\:\/\/localhost\:9200\"#window.location.protocol \+ \'\/\/\' \+ window.location.hostname \+ \'\:\' \+ window.location.port \+ \'\/es\/\'#" /usr/src/app/_site/app.js && \
 
 # Setup user, groups and configs
     addgroup -g 2000 head && \
-    adduser -S -H -s /bin/bash -u 2000 -D -g 2000 head && \
+    adduser -S -H -s /bin/ash -u 2000 -D -g 2000 head && \
     chown -R head:head /usr/src/app/ && \
 
 # Clean up
-    apk del git
+    apk del --purge git && \
+    rm -rf /root/* && \
+    rm -rf /tmp/* && \
+    rm -rf /var/cache/apk/*
 
 # Healthcheck
 HEALTHCHECK --retries=10 CMD curl -s -XGET 'http://127.0.0.1:9100'
 
 # Start elasticsearch-head
-USER head
+USER head:head
 WORKDIR /usr/src/app
 CMD ["node_modules/http-server/bin/http-server", "_site", "-p", "9100"]

docker/elk/head/docker-compose.yml

@@ -0,0 +1,16 @@
+version: '2.3'
+
+services:
+
+## Elasticsearch-head service
+  head:
+    build: .
+    container_name: head
+    restart: always
+    #    depends_on:
+    #      elasticsearch:
+    #        condition: service_healthy
+    ports:
+     - "127.0.0.1:64302:9100"
+    image: "dtagdevsec/head:1811"
+    read_only: true

docker/elk/kibana/Dockerfile

@@ -1,24 +1,23 @@
 FROM alpine
-MAINTAINER MO
 
 # Include dist
 ADD dist/ /root/dist/
 
 # Setup env and apt
-RUN apk -U upgrade && \
-    apk add bash curl nodejs procps wget && \
+RUN apk -U add \
+            curl \
+            nodejs \
+            wget && \
 
 # Get and install packages
     cd /root/dist/ && \
     mkdir -p /usr/share/kibana/ && \
-    wget https://artifacts.elastic.co/downloads/kibana/kibana-5.6.9-linux-x86_64.tar.gz && \
-    tar xvfz kibana-5.6.9-linux-x86_64.tar.gz --strip-components=1 -C /usr/share/kibana/ && \
+    wget https://artifacts.elastic.co/downloads/kibana/kibana-6.5.1-linux-x86_64.tar.gz && \
+    tar xvfz kibana-6.5.1-linux-x86_64.tar.gz --strip-components=1 -C /usr/share/kibana/ && \
 
 # Kibana's bundled node does not work in alpine
     rm /usr/share/kibana/node/bin/node && \
-    rm /usr/share/kibana/node/bin/npm && \
     ln -s /usr/bin/node /usr/share/kibana/node/bin/node && \
-    ln -s /usr/bin/npm /usr/share/kibana/node/bin/npm && \
 
 # Add and move files
     cd /root/dist/ && \
@@ -28,25 +27,45 @@ RUN apk -U upgrade && \
     cp elk.ico /usr/share/kibana/src/ui/public/assets/favicons/favicon-16x16.png && \
     cp elk.ico /usr/share/kibana/src/ui/public/assets/favicons/favicon-32x32.png && \
     cp create_kibana_index.js /usr/share/kibana/src/core_plugins/elasticsearch/lib/ && \
-    cd / && \
+
+# Setup plugins, rebuild bundle
+    #cd /usr/share/kibana/plugins && \
+    #wget https://github.com/dlumbrer/kbn_radar/releases/download/Kibana-6.X/kbn_radar.tar.gz && \
+    #wget https://github.com/dlumbrer/kbn_network/releases/download/6.0.X-1/network_vis.tar.gz && \
+    #tar xvfz kbn_radar.tar.gz && \
+    #tar xvfz network_vis.tar.gz && \
+    #rm *.tar.gz && \
+    rm -rf /usr/share/kibana/optimize/bundles/* && \
 
 # Setup user, groups and configs
     sed -i 's/#server.basePath: ""/server.basePath: "\/kibana"/' /usr/share/kibana/config/kibana.yml && \
-    sed -i 's/#kibana.defaultAppId: "discover"/kibana.defaultAppId: "dashboards"/' /usr/share/kibana/config/kibana.yml && \
+    sed -i 's/#kibana.defaultAppId: "home"/kibana.defaultAppId: "dashboards"/' /usr/share/kibana/config/kibana.yml && \
     sed -i 's/#server.host: "localhost"/server.host: "0.0.0.0"/' /usr/share/kibana/config/kibana.yml && \
     sed -i 's/#elasticsearch.url: "http:\/\/localhost:9200"/elasticsearch.url: "http:\/\/elasticsearch:9200"/' /usr/share/kibana/config/kibana.yml && \
-    /usr/share/kibana/bin/kibana 2>&1 | grep -m 1 "Optimization of bundles" && \
+    sed -i 's/#server.rewriteBasePath: false/server.rewriteBasePath: false/' /usr/share/kibana/config/kibana.yml && \
+    sed -i "s/#005571/#e20074/g" /usr/share/kibana/src/ui/public/chrome/directives/global_nav/global_nav.less && \
+    sed -i "s/globalColorBlue/globalColorMagenta/g" /usr/share/kibana/src/ui/public/chrome/directives/global_nav/global_nav_link/global_nav_link.less && \
+    echo "@globalColorMagenta: #9E0051;" >> /usr/share/kibana/src/ui/public/styles/variables/colors.less && \
+    echo "xpack.infra.enabled: false" >> /usr/share/kibana/config/kibana.yml && \ 
+    echo "xpack.logstash.enabled: false" >> /usr/share/kibana/config/kibana.yml && \
+    echo "xpack.canvas.enabled: false" >> /usr/share/kibana/config/kibana.yml && \
+    echo "xpack.spaces.enabled: false" >> /usr/share/kibana/config/kibana.yml && \
+    echo "xpack.apm.enabled: false" >> /usr/share/kibana/config/kibana.yml && \
+    /usr/share/kibana/bin/kibana --optimize && \
     addgroup -g 2000 kibana && \
-    adduser -S -H -s /bin/bash -u 2000 -D -g 2000 kibana && \
+    adduser -S -H -s /bin/ash -u 2000 -D -g 2000 kibana && \
     chown -R kibana:kibana /usr/share/kibana/ && \
 
 # Clean up
-    apk del wget && \
-    rm -rf /root/*
+    apk del --purge wget && \
+    rm -rf /root/* && \
+    rm -rf /tmp/* && \
+    rm -rf /var/cache/apk/*
 
 # Healthcheck
 HEALTHCHECK --retries=10 CMD curl -s -XGET 'http://127.0.0.1:5601'
 
 # Start kibana
-USER kibana
+STOPSIGNAL SIGKILL
+USER kibana:kibana
 CMD ["/usr/share/kibana/bin/kibana"]

docker/elk/kibana/docker-compose.yml

@@ -0,0 +1,15 @@
+version: '2.3'
+
+services:
+
+## Kibana service
+  kibana:
+    build: .
+    container_name: kibana
+    restart: always
+#    depends_on:
+#      elasticsearch:
+#        condition: service_healthy
+    ports:
+     - "127.0.0.1:64296:5601"
+    image: "dtagdevsec/kibana:1811"

docker/elk/logstash/Dockerfile

@@ -1,23 +1,28 @@
 FROM alpine
-MAINTAINER MO
 
 # Include dist
 ADD dist/ /root/dist/
 
 # Setup env and apt
-RUN apk -U upgrade && \
-    apk add bash curl git libc6-compat libzmq openjdk8-jre procps wget && \
+RUN apk -U add \
+             bash \
+             curl \
+             git \
+             libc6-compat \
+             libzmq \
+             openjdk8-jre \
+             wget && \
 
 # Get and install packages
     git clone --depth=1 https://github.com/dtag-dev-sec/listbot /etc/listbot && \
     cd /root/dist/ && \
     mkdir -p /usr/share/logstash/ && \
-    wget https://artifacts.elastic.co/downloads/logstash/logstash-5.6.9.tar.gz && \
+    wget https://artifacts.elastic.co/downloads/logstash/logstash-6.5.1.tar.gz && \
     wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-ASN.tar.gz && \
-    tar xvfz logstash-5.6.9.tar.gz --strip-components=1 -C /usr/share/logstash/ && \
+    tar xvfz logstash-6.5.1.tar.gz --strip-components=1 -C /usr/share/logstash/ && \
     /usr/share/logstash/bin/logstash-plugin install logstash-filter-translate && \
     /usr/share/logstash/bin/logstash-plugin install logstash-output-syslog && \
-    tar xvfz GeoLite2-ASN.tar.gz --strip-components=1 -C /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-geoip-4.3.1-java/vendor/ && \
+    tar xvfz GeoLite2-ASN.tar.gz --strip-components=1 -C /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/ && \
 
 # Add and move files
     cd /root/dist/ && \
@@ -25,14 +30,24 @@ RUN apk -U upgrade && \
     chmod u+x /usr/bin/update.sh && \
     mkdir -p /etc/logstash/conf.d && \
     cp logstash.conf /etc/logstash/conf.d/ && \
-    cp elasticsearch-template-es5x.json /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.4.2-java/lib/logstash/outputs/elasticsearch/ && \
+    cp elasticsearch-template-es6x.json /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.1-java/lib/logstash/outputs/elasticsearch/ && \
+
+# Setup user, groups and configs
+    addgroup -g 2000 logstash && \
+    adduser -S -H -s /bin/bash -u 2000 -D -g 2000 logstash && \
+    chown -R logstash:logstash /usr/share/logstash && \
+    chown -R logstash:logstash /etc/listbot && \
+    chmod 755 /usr/bin/update.sh && \
 
 # Clean up
-    apk del wget && \
-    rm -rf /root/*
+    apk del --purge wget && \
+    rm -rf /root/* && \
+    rm -rf /tmp/* && \
+    rm -rf /var/cache/apk/*
 
 # Healthcheck
 HEALTHCHECK --retries=10 CMD curl -s -XGET 'http://127.0.0.1:9600'
 
 # Start logstash
-CMD update.sh && /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf
+#USER logstash:logstash
+CMD update.sh && exec /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf --java-execution

docker/elk/logstash/dist/elasticsearch-template-es6x.json

@@ -0,0 +1,47 @@
+{
+  "template" : "logstash-*",
+  "version" : 60001,
+  "settings" : {
+    "index.refresh_interval" : "5s",
+    "index.number_of_shards" : "1",
+    "index.number_of_replicas" : "0"
+  },
+  "mappings" : {
+    "_default_" : {
+      "dynamic_templates" : [ {
+        "message_field" : {
+          "path_match" : "message",
+          "match_mapping_type" : "string",
+          "mapping" : {
+            "type" : "text",
+            "norms" : false
+          }
+        }
+      }, {
+        "string_fields" : {
+          "match" : "*",
+          "match_mapping_type" : "string",
+          "mapping" : {
+            "type" : "text", "norms" : false,
+            "fields" : {
+              "keyword" : { "type": "keyword", "ignore_above": 256 }
+            }
+          }
+        }
+      } ],
+      "properties" : {
+        "@timestamp": { "type": "date"},
+        "@version": { "type": "keyword"},
+        "geoip"  : {
+          "dynamic": true,
+          "properties" : {
+            "ip": { "type": "ip" },
+            "location" : { "type" : "geo_point" },
+            "latitude" : { "type" : "half_float" },
+            "longitude" : { "type" : "half_float" }
+          }
+        }
+      }
+    }
+  }
+}

docker/elk/logstash/dist/logstash.conf

@@ -15,9 +15,16 @@ input {
     type => "P0f"
   }
 
-# Conpot 
+# Ciscoasa
   file {
-    path => ["/data/conpot/log/conpot.json"]
+    path => ["/data/ciscoasa/log/ciscoasa.log"]
+    codec => plain
+    type => "Ciscoasa"
+  }
+
+# Conpot
+  file {
+    path => ["/data/conpot/log/*.json"]
     codec => json
     type => "ConPot"
   }
@@ -43,18 +50,25 @@ input {
     type => "ElasticPot"
   }
 
-# eMobility
-  file {
-    path => ["/data/emobility/log/centralsystemEWS.log"]
-    type => "eMobility"
-  }
-
 # Glastopf
   file {
     path => ["/data/glastopf/log/glastopf.log"]
     type => "Glastopf"
   }
 
+# Glutton
+  file {
+    path => ["/data/glutton/log/glutton.log"]
+    codec => json
+    type => "Glutton"
+  }
+
+# Heralding
+  file {
+    path => ["/data/heralding/log/auth.csv"]
+    type => "Heralding"
+  }
+
 # Honeytrap
   file {
     path => ["/data/honeytrap/log/attackers.json"]
@@ -62,37 +76,39 @@ input {
     type => "Honeytrap"
   }
 
-# Mailoney 
+# Mailoney
   file {
     path => ["/data/mailoney/log/commands.log"]
     type => "Mailoney"
   }
 
+# Medpot
+  file {
+    path => ["/data/medpot/log/medpot.log"]
+    codec => json
+    type => "Medpot"
+  }
+
 # Rdpy
   file {
     path => ["/data/rdpy/log/rdpy.log"]
     type => "Rdpy"
   }
 
-# Host Syslog
-  file {
-    path => ["/data/host/log/auth.log"]
-    codec => plain
-    type => "Syslog"
-  }
-
 # Host NGINX
   file {
-    path => ["/data/host/log/nginx/access.log"]
+    path => ["/data/nginx/log/access.log"]
     codec => json
     type => "NGINX"
   }
 
-# Vnclowpot
+# Tanner
   file {
-    path => ["/data/vnclowpot/log/vnclowpot.log"]
-    type => "Vnclowpot"
+    path => ["/data/tanner/log/tanner_report.json"]
+    codec => json
+    type => "Tanner"
   }
+
 }
 
 # Filter Section
@@ -127,11 +143,35 @@ filter {
     }
   }
 
+# Ciscoasa
+  if [type] == "Ciscoasa" {
+    kv {
+      remove_char_key => " '{}"
+      remove_char_value => "'{}"
+      value_split => ":"
+      field_split => ","
+    }
+    date {
+      match => [ "timestamp", "ISO8601" ]
+    }
+    mutate {
+      add_field => {
+        "dest_ip" => "${MY_EXTIP}"
+      }
+    }
+  }
+
 # Conpot
   if [type] == "ConPot" {
     date {
       match => [ "timestamp", "ISO8601" ]
     }
+    mutate { 
+      rename => { 
+        "dst_port" => "dest_port" 
+        "dst_ip" => "dest_ip" 
+      } 
+    } 
   }
 
 # Cowrie
@@ -165,8 +205,8 @@ filter {
     if [credentials] {
       mutate {
         add_field => {
-          "login.username" => "%{[credentials][username]}"
-          "login.password" => "%{[credentials][password]}"
+          "username" => "%{[credentials][username]}"
+          "password" => "%{[credentials][password]}"
         }
         remove_field => "[credentials]"
       }
@@ -180,16 +220,6 @@ filter {
     }
   }
 
-# eMobility
-  if [type] == "eMobility" {
-    grok {
-      match => [ "message", "\A%{IP:src_ip}\.%{POSINT:src_port:integer}\|%{IP:dest_ip}\.%{POSINT:dest_port:integer}:%{SPACE}%{SYSLOG5424PRINTASCII}%{SPACE}%{SYSLOG5424PRINTASCII}%{SPACE}%{SYSLOG5424PRINTASCII}%{SPACE}%{SYSLOG5424PRINTASCII}%{SPACE}%{SYSLOG5424PRINTASCII}%{SPACE}%{SYSLOG5424PRINTASCII}%{SPACE}%{SYSLOG5424PRINTASCII}%{SPACE}%{SYSLOG5424PRINTASCII}%{SPACE}%{SYSLOG5424PRINTASCII}%{SPACE}%{SYSLOG5424SD}%{SYSLOG5424PRINTASCII}%{SPACE}%{SYSLOG5424PRINTASCII}%{SPACE}%{SYSLOG5424PRINTASCII}%{SPACE}%{URIPROTO:http_method}\|%{URIPATH:http_uri}\|%{TIMESTAMP_ISO8601:timestamp}" ]
-    }
-    date {
-      match => [ "timestamp", "ISO8601" ]
-    }
-  }
-
 # Glastopf
   if [type] == "Glastopf" {
     grok {
@@ -201,6 +231,25 @@ filter {
     }
   }
 
+# Glutton
+  if [type] == "Glutton" {
+    date {
+      match => [ "ts", "UNIX" ]
+      remove_field => ["ts"]
+    }
+  }
+
+# Heralding
+  if [type] == "Heralding" {
+    csv {
+      columns => ["timestamp","auth_id","session_id","src_ip","src_port","dest_ip","dest_port","proto","username","password"] separator => ","
+    }
+    date {
+      match => [ "timestamp", "yyyy-MM-dd HH:mm:ss.SSSSSS" ]
+      remove_field => ["timestamp"]
+    }
+  }
+
 # Honeytrap
   if [type] == "Honeytrap" {
     date {
@@ -232,6 +281,19 @@ filter {
     }
   }
 
+# Medpot
+  if [type] == "Medpot" {
+    mutate {
+      add_field => {
+        "dest_port" => "2575"
+        "dest_ip" => "${MY_EXTIP}"
+      }
+    }
+    date {
+      match => [ "timestamp", "ISO8601" ]
+    }
+  }
+
 # Rdpy
   if [type] == "Rdpy" {
     grok { match => { "message" => [ "\A%{TIMESTAMP_ISO8601:timestamp},domain:%{CISCO_REASON:domain},username:%{CISCO_REASON:username},password:%{CISCO_REASON:password},hostname:%{GREEDYDATA:hostname}", "\A%{TIMESTAMP_ISO8601:timestamp},Connection from %{IPV4:src_ip}:%{INT:src_port:integer}" ] } }
@@ -246,79 +308,6 @@ filter {
     }
   }
 
-# Syslog
-  if [type] == "Syslog" {
-      grok {
-        match => {
-          "message" => ["%{SYSLOGPAMSESSION}", "%{CRONLOG}", "%{SYSLOGLINE}"]
-        }
-        overwrite => "message"
-      }
-      date {
-        match => [ "timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
-        remove_field => ["timestamp"]
-      }
-      date {
-        match => ["timestamp8601", "ISO8601"]
-        remove_field => ["timestamp8601"]
-      }
-      grok {
-        match => { "message" => "Connection closed by %{IP:src_ip}" }
-        add_tag => [ "ssh_connection_closed" ]
-        tag_on_failure => []
-      }
-      grok {
-        match => { "message" => "Received disconnect from %{IP:src_ip}" }
-        add_tag => [ "ssh_connection_disconnect" ]
-        tag_on_failure => []
-      }
-      grok {
-        match => { "message" => "Failed password for invalid user %{USERNAME:username} from %{IP:src_ip} port %{BASE10NUM:port} ssh2" }
-        add_tag => [ "ssh_failed_password" ]
-        tag_on_failure => []
-      }
-      grok {
-        match => { "message" => "Did not receive identification string from %{IP:src_ip}" }
-        add_tag => [ "ssh_no_id" ]
-        tag_on_failure => []
-      }
-      grok {
-        match => { "message" => "User %{USERNAME:username} from %{IP:src_ip} not allowed because not listed in AllowUsers" }
-        add_tag => [ "ssh_user_not_allowed" ]
-        tag_on_failure => []
-      }
-      grok {
-        match => { "message" => "authentication failure; logname=%{USERNAME:logname} uid=%{BASE10NUM:uid} euid=%{BASE10NUM:euid} tty=%{TTY:tty} ruser=%{USERNAME:ruser} rhost=(?:%{HOSTNAME:remote_host}|\s*) user=%{USERNAME:user}"}
-        add_tag => [ "ssh_auth_failure" ]
-        tag_on_failure => []
-      }
-      grok {
-        match => { "message" => "pam_unix\(sshd:auth\): authentication failure; logname= uid=0 euid=0 tty=%{NOTSPACE:tty} ruser= rhost=(?:%{HOSTNAME:remote_host}|\s*)  user=%{USERNAME:user}"}
-        add_tag => [ "ssh_auth_failure" ]
-        tag_on_failure => []
-      }
-      grok {
-        match => { "message" => "Failed password for %{USERNAME:username} from %{IP:src_ip} port %{BASE10NUM:port} ssh2"}
-        add_tag => [ "ssh_failed_password" ]
-        tag_on_failure => []
-      }
-      grok {
-        match => { "message" => "Accepted password for %{USERNAME:username} from %{IP:src_ip} port %{BASE10NUM:port} ssh2"}
-        add_tag => [ "ssh_accepted_password" ]
-        tag_on_failure => []
-      }
-      grok {
-        match => { "message" => "Accepted publickey for %{USERNAME:username} from %{IP:src_ip} port %{BASE10NUM:port} ssh2"}
-        add_tag => [ "ssh_accepted_pubkey" ]
-        tag_on_failure => []
-      }
-      grok {
-        match => { "message" => "Accepted keyboard-interactive/pam for %{USERNAME:username} from %{IP:src_ip} port %{BASE10NUM:port} ssh2"}
-        add_tag => [ "ssh_accepted_interactive" ]
-        tag_on_failure => []
-      }
-  }
-
 # NGINX
   if [type] == "NGINX" {
     date {
@@ -326,18 +315,18 @@ filter {
     }
   }
 
-# Vnclowpot 
-  if [type] == "Vnclowpot" {
-    grok {
-      match => [ "message", "\A%{NOTSPACE}%{SPACE}%{TIME}%{SPACE}%{IPV4:src_ip}:%{INT:src_port}%{SPACE}%{NOTSPACE:vnc_handshake}" ]
-    }
+# Tanner
+  if [type] == "Tanner" {
     date {
-      match => [ "timestamp", "yyyy/MM/dd HH:mm:ss" ]
-      remove_field => ["timestamp"]
+      match => [ "timestamp", "ISO8601" ]
     }
     mutate {
+      rename => {
+        "[peer][ip]" => "src_ip"
+        "[peer][port]" => "src_port"
+      }
       add_field => {
-        "dest_port" => "5900"
+        "dest_port" => "80"
       }
     }
   }
@@ -350,22 +339,22 @@ if "_grokparsefailure" in [tags] { drop {} }
     geoip {
       cache_size => 10000
       source => "src_ip"
-      database => "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-geoip-4.3.1-java/vendor/GeoLite2-City.mmdb"
-    } 
+      database => "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-City.mmdb"
+    }
     geoip {
       cache_size => 10000
       source => "src_ip"
-      database => "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-geoip-4.3.1-java/vendor/GeoLite2-ASN.mmdb"
-    } 
+      database => "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-ASN.mmdb"
+    }
     translate {
       refresh_interval => 86400
       field => "src_ip"
       destination => "ip_rep"
       dictionary_path => "/etc/listbot/iprep.yaml"
     }
-  } 
+  }
 
-# In some rare conditions dest_port, src_port is indexed as string, forcing integer for now
+# In some rare conditions dest_port, src_port, status are indexed as string, forcing integer for now
   if [dest_port] {
     mutate {
         convert => { "dest_port" => "integer" }
@@ -376,9 +365,14 @@ if "_grokparsefailure" in [tags] { drop {} }
         convert => { "src_port" => "integer" }
     }
   }
+  if [status] {
+    mutate {
+        convert => { "status" => "integer" }
+    }
+  }
 
 # Add T-Pot hostname and external IP
-  if [type] == "ConPot" or [type] == "Cowrie" or [type] == "Dionaea" or [type] == "ElasticPot" or [type] == "eMobility" or [type] == "Glastopf" or [type] == "Honeytrap" or [type] == "Mailoney" or [type] == "Rdpy" or [type] == "Suricata" or [type] == "Vnclowpot" {
+  if [type] == "Ciscoasa" or [type] == "ConPot" or [type] == "Cowrie" or [type] == "Dionaea" or [type] == "ElasticPot" or [type] == "Glastopf" or [type] == "Glutton" or [type] == "Honeytrap" or [type] == "Heralding" or [type] == "Mailoney" or [type] == "Medpot" or [type] == "P0f" or [type] == "Rdpy" or [type] == "Suricata" or [type] == "Tanner" {
     mutate {
       add_field => {
         "t-pot_ip_ext" => "${MY_EXTIP}"
@@ -394,6 +388,7 @@ if "_grokparsefailure" in [tags] { drop {} }
 output {
   elasticsearch {
     hosts => ["elasticsearch:9200"]
+#    document_type => "doc"
   }
 
   if [type] == "Suricata" {

docker/elk/logstash/docker-compose.yml

@@ -0,0 +1,18 @@
+version: '2.3'
+
+services:
+
+## Logstash service
+  logstash:
+    build: .
+    container_name: logstash
+    restart: always
+#    depends_on:
+#      elasticsearch:
+#        condition: service_healthy
+    env_file:
+     - /opt/tpot/etc/compose/elk_environment
+    image: "dtagdevsec/logstash:1811"
+    volumes:
+     - /data:/data
+     - /root/tpotce/docker/elk/logstash/dist/logstash.conf:/etc/logstash/conf.d/logstash.conf

docker/ews/Dockerfile

@@ -1,32 +1,53 @@
 FROM alpine
-MAINTAINER MO
 
 # Include dist
 ADD dist/ /root/dist/
 
 # Install packages
-RUN apk -U upgrade && \
-    apk add build-base git libssl1.0 openssl-dev python-dev py-cffi py-ipaddress py-lxml py-mysqldb py-pip py-pysqlite py-requests py-setuptools && \
-    pip install pyOpenSSL==16.2.0 && \
+RUN apk -U --no-cache add \
+            build-base \
+            git \
+            libffi-dev \
+            libssl1.0 \
+            openssl-dev \
+            python-dev \
+            py-cffi \
+            py-ipaddress \
+            py-lxml \
+            py-mysqldb \
+            py-pip \
+            py-pysqlite \
+            py-requests \
+            py-setuptools && \
+    pip install --no-cache-dir -U pip && \
+    pip install --no-cache-dir pyOpenSSL && \
 
 # Setup ewsposter
-    git clone https://github.com/rep/hpfeeds /opt/hpfeeds && \
+    git clone --depth=1 https://github.com/rep/hpfeeds /opt/hpfeeds && \
     cd /opt/hpfeeds && \
     python setup.py install && \
-    git clone https://github.com/vorband/ewsposter /opt/ewsposter && \
+    git clone --depth=1 https://github.com/vorband/ewsposter /opt/ewsposter && \
     mkdir -p /opt/ewsposter/spool /opt/ewsposter/log && \
 
 # Setup user and groups
     addgroup -g 2000 ews && \
     adduser -S -H -u 2000 -D -g 2000 ews && \
+    chown -R ews:ews /opt/ewsposter && \
 
 # Supply configs
     mv /root/dist/ews.cfg /opt/ewsposter/ && \
 
 # Clean up
-    apk del build-base git openssl-dev python-dev py-pip py-setuptools && \
+    apk del build-base \
+            git \
+            openssl-dev \
+            python-dev \
+            py-pip \
+            py-setuptools && \
     rm -rf /root/* && \
     rm -rf /var/cache/apk/*
 
 # Run ewsposter
-CMD sleep 10 && /usr/bin/python /opt/ewsposter/ews.py -l 60
+STOPSIGNAL SIGINT
+USER ews:ews
+CMD sleep 10 && exec /usr/bin/python -u /opt/ewsposter/ews.py -l 60

docker/ews/README.md

@@ -1,4 +1,4 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/ewsposter:1710.svg)](https://microbadger.com/images/dtagdevsec/ewsposter:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/ewsposter:1710.svg)](https://microbadger.com/images/dtagdevsec/ewsposter:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/ewsposter:1811.svg)](https://microbadger.com/images/dtagdevsec/ewsposter:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/ewsposter:1811.svg)](https://microbadger.com/images/dtagdevsec/ewsposter:1811 "Get your own image badge on microbadger.com")
 
 # ewsposter
 

docker/ews/dist/ews.cfg

@@ -46,12 +46,12 @@ malwaredir =
 
 [KIPPO]
 kippo = false
-nodeid = 
-mysqlhost = 
-mysqldb = 
-mysqluser = 
-mysqlpw = 
-malwaredir = 
+nodeid =
+mysqlhost =
+mysqldb =
+mysqluser =
+mysqlpw =
+malwaredir =
 
 [COWRIE]
 cowrie = true
@@ -78,14 +78,14 @@ iptableslog =
 targetip =
 
 [EMOBILITY]
-eMobility = true
+eMobility = false
 nodeid = emobility-community-01
 logfile = /data/emobility/log/centralsystemEWS.log
 
 [CONPOT]
 conpot = true
 nodeid = conpot-community-01
-logfile = /data/conpot/log/conpot.json
+logfile = /data/conpot/log/conpot*.json
 
 [ELASTICPOT]
 elasticpot = true
@@ -111,3 +111,18 @@ logfile = /data/rdpy/log/rdpy.log
 vnclowpot = true
 nodeid = vnclowpot-community-01
 logfile = /data/vnclowpot/log/vnclowpot.log
+
+[HERALDING]
+heralding = true
+nodeid = heralding-community-01
+logfile = /data/heralding/log/auth.csv
+
+[CISCOASA]
+ciscoasa = true
+nodeid = ciscoasa-community-01
+logfile = /data/ciscoasa/log/ciscoasa.log
+
+[TANNER]
+tanner = true
+nodeid = tanner-community-01
+logfile = /data/tanner/log/tanner_report.json

docker/ews/docker-compose.yml

@@ -0,0 +1,20 @@
+version: '2.3'
+
+networks:
+  ewsposter_local:
+
+services:
+
+# Ewsposter service
+  ewsposter:
+    build: .
+    container_name: ewsposter
+    restart: always
+    networks:
+     - ewsposter_local
+    env_file:
+     - /opt/tpot/etc/compose/elk_environment
+    image: "dtagdevsec/ewsposter:1811"
+    volumes:
+     - /data:/data
+     - /data/ews/conf/ews.ip:/opt/ewsposter/ews.ip

docker/glastopf/Dockerfile

@@ -1,21 +1,32 @@
-FROM alpine 
-MAINTAINER MO 
+FROM alpine
 
 # Include dist
 ADD dist/ /root/dist/
 
 # Install packages
-RUN apk -U upgrade && \
-    apk add autoconf bash bind-tools build-base cython git libffi libffi-dev make py-asn1 \
-            py-cffi py-chardet py-chardet py-cparser py-cryptography py-dateutil \
-            py-enum34 py-idna py-ipaddress py-jinja2 py-lxml py-mysqldb py-openssl \
-            py-pip py-requests py-setuptools python python-dev && \
-    apk -U add --repository http://dl-3.alpinelinux.org/alpine/edge/testing/ \
-            py-beautifulsoup4 php7 php7-dev py-cssselect py-gevent py-greenlet py-mongo \
-            py-sqlalchemy py-webob && \
+RUN apk -U --no-cache add \
+               autoconf \
+               bind-tools \
+               build-base \
+               cython \
+               git \
+               libffi \
+               libffi-dev \
+               libcap \
+               libxslt-dev \
+               make \
+               php7 \
+               php7-dev \
+               py-mysqldb \
+               py-openssl \
+               py-pip \
+               py-setuptools \
+               python \
+               python-dev && \
+    pip install --no-cache-dir --upgrade pip && \
 
 # Install php sandbox from git
-    git clone https://github.com/glastopf/BFR.git /opt/BFR && \
+    git clone --depth=1 https://github.com/mushorg/BFR /opt/BFR && \
     cd /opt/BFR && \
     phpize7 && \
     ./configure \
@@ -28,23 +39,34 @@ RUN apk -U upgrade && \
     echo "zend_extension = "$(find /usr -name bfr.so) >> /etc/php7/php.ini && \
 
 # Install glastopf from git
-    git clone https://github.com/mushorg/glastopf.git /opt/glastopf && \
+    git clone --depth=1 https://github.com/mushorg/glastopf.git /opt/glastopf && \
     cd /opt/glastopf && \
-    python setup.py install && \
+    cp /root/dist/requirements.txt . && \
+    pip install --no-cache-dir . && \
     cd / && \
     rm -rf /opt/glastopf /tmp/* /var/tmp/* && \
+    setcap cap_net_bind_service=+ep /usr/bin/python2.7 && \
 
 # Setup user, groups and configs
     addgroup -g 2000 glastopf && \
     adduser -S -H -u 2000 -D -g 2000 glastopf && \
-    mkdir -p /opt/glastopf && \
-    mv /root/dist/glastopf.cfg /opt/glastopf/ && \
+    mkdir -p /etc/glastopf && \
+    mv /root/dist/glastopf.cfg /etc/glastopf/ && \
 
 # Clean up
-    apk del autoconf build-base git libffi-dev php7-dev python-dev && \
+    apk del --purge autoconf \
+                    build-base \
+                    file \
+                    git \
+                    libffi-dev \
+                    php7-dev \
+                    python-dev \
+                    py-pip && \
     rm -rf /root/* && \
     rm -rf /var/cache/apk/*
 
 # Set workdir and start glastopf
-WORKDIR /opt/glastopf/
-CMD ["glastopf-runner"]
+STOPSIGNAL SIGINT
+USER glastopf:glastopf
+WORKDIR /tmp/glastopf/
+CMD cp /etc/glastopf/glastopf.cfg /tmp/glastopf && exec glastopf-runner

docker/glastopf/README.md

@@ -1,8 +1,8 @@
-[![](https://images.microbadger.com/badges/version/dtagdevsec/glastopf:1710.svg)](https://microbadger.com/images/dtagdevsec/glastopf:1710 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/glastopf:1710.svg)](https://microbadger.com/images/dtagdevsec/glastopf:1710 "Get your own image badge on microbadger.com")
+[![](https://images.microbadger.com/badges/version/dtagdevsec/glastopf:1811.svg)](https://microbadger.com/images/dtagdevsec/glastopf:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/glastopf:1811.svg)](https://microbadger.com/images/dtagdevsec/glastopf:1811 "Get your own image badge on microbadger.com")
 
 # glastopf
 
-[glastopf](https://github.com/glastopf/glastopf) is a python web application honeypot.
+[glastopf](https://github.com/mushorg/glastopf) is a python web application honeypot.
 
 This dockerized version is part of the **[T-Pot community honeypot](http://dtag-dev-sec.github.io/)** of Deutsche Telekom AG.
 
@@ -12,4 +12,4 @@ The `docker-compose.yml` contains the necessary settings to test glastopf using
 
 # Glastopf Dashboard
 
-![Glastopf Dashboard](https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/docker/glastopf/doc/dashboard.png)
+![Glastopf Dashboard](doc/dashboard.png)

docker/glastopf/dist/glastopf.cfg

@@ -104,3 +104,12 @@ sensorid = None
 
 [profiler]
 enabled = False
+
+[s3storage]
+enabled = False
+endpoint = http://localhost:8080/
+aws_access_key_id = YOUR_aws_access_key_id
+aws_secret_access_key = YOUR_aws_access_key_id
+bucket = glastopf
+region = eu-west-1
+signature_version = s3

docker/glastopf/dist/requirements.txt

@@ -0,0 +1,35 @@
+asn1crypto==0.24.0
+BeautifulSoup==3.2.1
+beautifulsoup4==4.6.1
+botocore==1.11.6
+certifi==2018.4.16
+cffi==1.10.0
+chardet==3.0.4
+cryptography==2.1.4
+cssselect==1.0.0
+Cython==0.28.2
+docutils==0.14
+enum34==1.1.6
+gevent==1.2.2
+greenlet==0.4.13
+hpfeeds==1.0
+idna==2.6
+ipaddress==1.0.22
+Jinja2==2.9.6
+jmespath==0.9.3
+libtaxii==1.1.111
+lxml==4.2.4
+MarkupSafe==1.0
+MySQL-python==1.2.5
+pyasn1==0.4.2
+pycparser==2.18
+pylibinjection==0.2.4
+pymongo==3.2.2
+pyOpenSSL==17.2.0
+python-dateutil==2.6.1
+python-logstash==0.4.6
+requests==2.18.4
+six==1.11.0
+SQLAlchemy==1.2.7
+urllib3==1.22
+WebOb==1.2.3

docker/glastopf/docker-compose.yml

@@ -1,4 +1,4 @@
-version: '2.1'
+version: '2.3'
 
 networks:
   glastopf_local:
@@ -7,13 +7,17 @@ services:
 
 # Glastopf service
   glastopf:
+    build: .
     container_name: glastopf
+    tmpfs:
+      - /tmp/glastopf:uid=2000,gid=2000
     restart: always
     networks:
      - glastopf_local
     ports:
      - "80:80"
-    image: "dtagdevsec/glastopf:1710"
+    image: "dtagdevsec/glastopf:1811"
+    read_only: true
     volumes:
-     - /data/glastopf/db:/opt/glastopf/db
-     - /data/glastopf/log:/opt/glastopf/log
+     - /data/glastopf/db:/tmp/glastopf/db
+     - /data/glastopf/log:/tmp/glastopf/log

docker/glutton/Dockerfile

@@ -0,0 +1,54 @@
+FROM alpine
+
+# Include dist
+ADD dist/ /root/dist/
+  
+# Setup apk
+RUN apk -U --no-cache add \
+                   build-base \
+                   git \
+                   go \
+                   g++ \
+                   iptables-dev \
+                   libnetfilter_queue-dev \
+                   libcap \
+                   libpcap-dev && \
+
+# Setup go, glutton
+    export GOPATH=/opt/go/ && \
+    go get -d github.com/mushorg/glutton && \
+    cd /opt/go/src/github.com/satori/ && \
+    rm -rf go.uuid && \ 
+    git clone https://github.com/satori/go.uuid && \
+    cd go.uuid && \
+    git checkout v1.2.0 && \
+    mv /root/dist/system.go /opt/go/src/github.com/mushorg/glutton/ && \
+    cd /opt/go/src/github.com/mushorg/glutton/ && \
+    make build && \
+    cd / && \
+    mkdir -p /opt/glutton && \
+    mv /opt/go/src/github.com/mushorg/glutton/bin /opt/glutton/ && \
+    mv /opt/go/src/github.com/mushorg/glutton/config /opt/glutton/ && \
+    mv /opt/go/src/github.com/mushorg/glutton/rules /opt/glutton/ && \
+    setcap cap_net_admin,cap_net_raw=+ep /opt/glutton/bin/server && \
+    setcap cap_net_admin,cap_net_raw=+ep /sbin/xtables-multi && \
+
+# Setup user, groups and configs
+    addgroup -g 2000 glutton && \
+    adduser -S -s /bin/ash -u 2000 -D -g 2000 glutton && \
+    mkdir -p /var/log/glutton && \
+    mv /root/dist/rules.yaml /opt/glutton/rules/ && \
+
+# Clean up
+    apk del --purge build-base \
+                    git \
+                    go \
+                    g++ && \
+    rm -rf /var/cache/apk/* \
+           /opt/go \
+           /root/dist
+
+# Start glutton 
+WORKDIR /opt/glutton
+USER glutton:glutton
+CMD exec bin/server -i $(/sbin/ip address | grep '^2: ' | awk '{ print $2 }' | tr -d [:punct:]) -l /var/log/glutton/glutton.log

docker/glutton/README.md

@@ -0,0 +1,15 @@
+[![](https://images.microbadger.com/badges/version/dtagdevsec/glutton:1811.svg)](https://microbadger.com/images/dtagdevsec/glutton:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/glutton:1811.svg)](https://microbadger.com/images/dtagdevsec/glutton:1811 "Get your own image badge on microbadger.com")
+
+# glutton
+
+[glutton](https://github.com/mushorg/glutton) is the all eating honeypot.
+
+This dockerized version is part of the **[T-Pot community honeypot](http://dtag-dev-sec.github.io/)** of Deutsche Telekom AG.
+
+The `Dockerfile` contains the blueprint for the dockerized glutton and will be used to setup the docker image.
+
+The `docker-compose.yml` contains the necessary settings to test glutton using `docker-compose`. This will ensure to start the docker container with the appropriate permissions and port mappings.  
+
+# glutton Dashboard
+
+![glutton Dashboard](doc/dashboard.png)

docker/glutton/dist/rules.yaml

@@ -0,0 +1,33 @@
+# Put passthrough rules on top, drop rules on bottom, rules are applied in order (top down)
+rules:
+  - match: udp dst port 53
+    type: passthrough
+  - match: tcp dst port 21
+    type: conn_handler
+    target: ftp
+  - match: tcp dst port 23 or port 2323 or port 23231
+    type: conn_handler
+    target: telnet
+  - match: tcp dst port 25
+    type: conn_handler
+    target: smtp
+  - match: tcp dst port 445
+    type: conn_handler
+    target: smb
+  - match: tcp dst port 1883
+    type: conn_handler
+    target: mqtt
+  - match: tcp dst port 3389
+    type: conn_handler
+    target: rdp
+  - match: tcp dst port 5060
+    type: conn_handler
+    target: sip
+  - match: tcp dst port 5222 or port 5223
+    type: conn_handler
+    target: jabber
+  - match: tcp
+    type: conn_handler
+    target: default
+  - match: 
+    type: drop

docker/glutton/dist/system.go

@@ -0,0 +1,38 @@
+package glutton
+
+import (
+	"fmt"
+	"log"
+	"os"
+	"os/exec"
+	"runtime"
+	"strings"
+	"time"
+)
+
+func countOpenFiles() int {
+	out, err := exec.Command("/bin/sh", "-c", fmt.Sprintf("lsof -p %v", os.Getpid())).Output()
+	if err != nil {
+		log.Fatal(err)
+	}
+	lines := strings.Split(string(out), "\n")
+	return len(lines) - 1
+}
+
+func countRunningRoutines() int {
+	return runtime.NumGoroutine()
+}
+
+func (g *Glutton) startMonitor(quit chan struct{}) {
+	ticker := time.NewTicker(10 * time.Second)
+	go func() {
+		for {
+			select {
+			case <-quit:
+				g.logger.Info("[system  ] Monitoring stopped..")
+				ticker.Stop()
+				return
+			}
+		}
+	}()
+}

docker/glutton/docker-compose.yml

@@ -0,0 +1,19 @@
+version: '2.3'
+
+services:
+
+# glutton service
+  glutton:
+    build: .
+    container_name: glutton
+    restart: always
+    tmpfs:
+     - /var/lib/glutton:uid=2000,gid=2000
+    network_mode: "host"
+    cap_add:
+     - NET_ADMIN
+    image: "dtagdevsec/glutton:1811"
+    read_only: true
+    volumes:
+     - /data/glutton/log:/var/log/glutton
+     - /root/tpotce/docker/glutton/dist/rules.yaml:/opt/glutton/rules/rules.yaml

docker/heralding/Dockerfile

@@ -0,0 +1,54 @@
+FROM alpine
+  
+# Include dist
+ADD dist/ /root/dist/
+
+# Install packages
+RUN apk -U --no-cache add \ 
+            		build-base \
+            		git \
+            		libcap \
+            		libffi-dev \
+            		libressl-dev \
+                        libzmq \
+            		postgresql-dev \
+            		python3 \
+            		python3-dev \
+            		py-virtualenv && \
+    pip3 install --no-cache-dir --upgrade pip && \
+
+# Setup heralding
+    mkdir -p /opt && \
+    cd /opt/ && \
+    git clone --depth=1 https://github.com/johnnykv/heralding && \
+    cd heralding && \
+    pip3 install --no-cache-dir -r requirements.txt && \
+    pip3 install --no-cache-dir . && \
+
+# Setup user, groups and configs
+    addgroup -g 2000 heralding && \
+    adduser -S -H -s /bin/ash -u 2000 -D -g 2000 heralding && \
+    mkdir -p /var/log/heralding/ /etc/heralding && \
+    mv /root/dist/heralding.yml /etc/heralding/ && \
+    setcap cap_net_bind_service=+ep /usr/bin/python3.6 && \
+    chown -R heralding:heralding /var/log/heralding && \
+
+# Clean up
+    apk del --purge \
+            build-base \
+            git \
+            libcap \
+            libffi-dev \
+            libressl-dev \
+            postgresql-dev \
+            python3-dev \
+            py-virtualenv && \
+    rm -rf /root/* \
+           /var/cache/apk/* \
+           /opt/heralding
+
+# Start elasticpot
+STOPSIGNAL SIGINT
+WORKDIR /tmp/heralding/
+USER heralding:heralding
+CMD exec heralding -c /etc/heralding/heralding.yml -l /var/log/heralding/heralding.log

docker/heralding/README.md

@@ -0,0 +1,15 @@
+[![](https://images.microbadger.com/badges/version/dtagdevsec/heralding:1811.svg)](https://microbadger.com/images/dtagdevsec/heralding:1811 "Get your own version badge on microbadger.com") [![](https://images.microbadger.com/badges/image/dtagdevsec/heralding:1811.svg)](https://microbadger.com/images/dtagdevsec/heralding:1811 "Get your own image badge on microbadger.com")
+
+# heralding
+
+[Heralding](https://github.com/johnnykv/heralding) is a simple honeypot that collects credentials, nothing more. Heralding is that honeypot! Currently the following protocols are supported: ftp, telnet, ssh, http, https, pop3, pop3s, imap, imaps, smtp and postgresql.
+
+This dockerized version is part of the **[T-Pot community honeypot](http://dtag-dev-sec.github.io/)** of Deutsche Telekom AG.
+
+The `Dockerfile` contains the blueprint for the dockerized heralding and will be used to setup the docker image.
+
+The `docker-compose.yml` contains the necessary settings to test conpot using `docker-compose`. This will ensure to start the docker container with the appropriate permissions and port mappings.
+
+# heralding Dashboard
+
+![heralding Dashboard](doc/dashboard.png)

docker/heralding/dist/heralding.yml

@@ -0,0 +1,152 @@
+# will request and log the public ip every hours from ipify
+public_ip_as_destination_ip: true
+
+# ip address to listen on
+bind_host: 0.0.0.0
+
+# logging of sessions and authentication attempts 
+activity_logging:
+  file:
+    enabled: true
+    session_log_file: "/var/log/heralding/session.csv"
+    authentication_log_file: "/var/log/heralding/auth.csv"
+
+  syslog:
+    enabled: false
+
+  hpfeeds:
+    enabled: false
+    session_channel: "heralding.session"
+    auth_channel: "heralding.auth"
+    host: 
+    port: 20000
+    ident:
+    secret:
+
+  curiosum:
+    enabled: false
+    port: 23400
+
+# protocols to enable
+capabilities:
+  ftp:
+    enabled: true
+    port: 21
+    timeout: 30
+    protocol_specific_data:
+      max_attempts: 3
+      banner: "Microsoft FTP Server"
+      syst_type: "Windows-NT"
+
+  telnet:
+    enabled: true
+    port: 23
+    timeout: 30
+    protocol_specific_data:
+      max_attempts: 3
+
+  pop3:
+    enabled: true
+    port: 110
+    timeout: 30
+    protocol_specific_data:
+      max_attempts: 3
+
+  pop3s:
+    enabled: true
+    port: 995
+    timeout: 30
+    protocol_specific_data:
+      max_attempts: 3
+      # if a .pem file is not found in work dir, a new pem file will be created
+      # using these values
+      cert:
+        common_name: "*"
+        country: "US"
+        state: None
+        locality: None
+        organization: None
+        organizational_unit: None
+        # how many days should the certificate be valid for
+        valid_days: 365
+        serial_number: 0
+
+  postgresql:
+    enabled: true
+    port: 5432
+    timeout: 30
+ 
+  imap:
+    enabled: true
+    port: 143
+    timeout: 30
+    protocol_specific_data:
+      max_attempts: 3
+      banner: "* OK IMAP4rev1 Server Ready"
+
+  imaps:
+    enabled: true
+    port: 993
+    timeout: 30
+    protocol_specific_data:
+      max_attempts: 3
+      banner: "* OK IMAP4rev1 Server Ready"
+      # if a .pem file is not found in work dir, a new pem file will be created
+      # using these values
+      cert:
+        common_name: "*"
+        country: "US"
+        state: None
+        locality: None
+        organization: None
+        organizational_unit: None
+        # how many days should the certificate be valid for
+        valid_days: 365
+        serial_number: 0
+
+  ssh:
+    enabled: true
+    port: 22
+    timeout: 30
+    protocol_specific_data:
+      banner: "SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.8"
+
+  http:
+    enabled: true
+    port: 80
+    timeout: 30
+    protocol_specific_data:
+      banner: ""
+
+  https:
+    enabled: true
+    port: 443
+    timeout: 30
+    protocol_specific_data:
+      banner: ""
+      # if a .pem file is not found in work dir, a new pem file will be created
+      # using these values
+      cert:
+        common_name: "*"
+        country: "US"
+        state: None
+        locality: None
+        organization: None
+        organizational_unit: None
+        # how many days should the certificate be valid for
+        valid_days: 365
+        serial_number: 0
+
+  smtp:
+    enabled: true
+    port: 25
+    timeout: 30
+    protocol_specific_data:
+      banner: "Microsoft ESMTP MAIL service ready"
+      # If the fqdn option is commented out or empty, then fqdn of the host will be used
+      fqdn: ""
+
+  vnc:
+    enabled: true
+    port: 5900
+    timeout: 30

docker/heralding/docker-compose.yml

@@ -0,0 +1,33 @@
+version: '2.3'
+  
+networks:
+  heralding_local:
+
+services:
+
+# Heralding service
+  heralding:
+    build: .
+    container_name: heralding
+    restart: always
+    tmpfs:
+     - /tmp/heralding:uid=2000,gid=2000
+    networks:
+     - heralding_local
+    ports:
+     - "21:21"
+     - "22:22"
+     - "23:23"
+     - "25:25"
+     - "80:80"
+     - "110:110"
+     - "143:143"
+     - "443:443"
+     - "993:993"
+     - "995:995"
+     - "5432:5432"
+     - "5900:5900"
+    image: "dtagdevsec/heralding:1811"
+    read_only: true
+    volumes:
+     - /data/heralding/log:/var/log/heralding