Alex-Devera/tpotce
1
0
Fork 0
mirror of https://github.com/telekom-security/tpotce.git synced 2025-04-29 19:58:52 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Merge pull request #403 from TheHADILP/all-ansible

Browse source 
Updated Ansible Deployment

@TheHADILP Thank you 😃
This commit is contained in:
Marco Ochse 2019-08-02 17:33:38 +02:00 committed by GitHub
commit 472edc5ac2
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
32 changed files with 436 additions and 467 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
CHANGELOG.md
View file

@ -1,5 +1,12 @@
# Changelog # Changelog
## 20190701
- **Reworked Ansible T-Pot Deployment**
- Transitioned from bash script to all Ansible
- Reusable Ansible Playbook for OpenStack clouds
- Example Showcase with our Open Telekom Cloud
- Adaptable for other cloud providers
## 20190626 ## 20190626
- **HPFEEDS Opt-In commandline option** - **HPFEEDS Opt-In commandline option**
- Pass a hpfeeds config file as a commandline argument - Pass a hpfeeds config file as a commandline argument

32
README.md
View file

@ -53,7 +53,7 @@ Furthermore we use the following tools
- [Post Install User](#postinstall) - [Post Install User](#postinstall)
- [Post Install Auto](#postinstallauto) - [Post Install Auto](#postinstallauto)
- [Cloud Deployments](#cloud) - [Cloud Deployments](#cloud)
- [Ansible Deployment on Open Telekom Cloud](#ansible-otc) - [Ansible](#ansible)
- [Terraform](#terraform) - [Terraform](#terraform)
- [First Run](#firstrun) - [First Run](#firstrun)
- [System Placement](#placement) - [System Placement](#placement)
@ -118,15 +118,16 @@ Furthermore we use the following tools
- **Fix #332** - **Fix #332**
- If T-Pot, opposed to the requirements, does not have full internet access netselect-apt fails to determine the fastest mirror as it needs ICMP and UDP outgoing. Should netselect-apt fail the default mirrors will be used. - If T-Pot, opposed to the requirements, does not have full internet access netselect-apt fails to determine the fastest mirror as it needs ICMP and UDP outgoing. Should netselect-apt fail the default mirrors will be used.
- **Improve install speed with apt-fast** - **Improve install speed with apt-fast**
- Migrating from a stable base install to Debian (Sid) requires downloading lots of packages. Depending on your geo location the download speed was already improved by introducing netselect-apt to determine the fastest mirror. Wit - Migrating from a stable base install to Debian (Sid) requires downloading lots of packages. Depending on your geo location the download speed was already improved by introducing netselect-apt to determine the fastest mirror. With apt-fast the downloads will be even faster by downloading packages not only in parallel but also with multiple connections per package.
h apt-fast the downloads will be even faster by downloading packages not only in parallel but also with multiple connections per package.
- **Added Ansible T-Pot Deployment on Open Telekom Cloud**
- Reusable Ansible Playbooks for all cloud providers
- Example Showcase with our Open Telekom Cloud
- **HPFEEDS Opt-In commandline option** - **HPFEEDS Opt-In commandline option**
- Pass a hpfeeds config file as a commandline argument - Pass a hpfeeds config file as a commandline argument
- hpfeeds config is saved in `/data/ews/conf/hpfeeds.cfg` - hpfeeds config is saved in `/data/ews/conf/hpfeeds.cfg`
- Update script restores hpfeeds config - Update script restores hpfeeds config
- **Ansible T-Pot Deployment**
- Transitioned from bash script to all Ansible
- Reusable Ansible Playbook for OpenStack clouds
- Example Showcase with our Open Telekom Cloud
- Adaptable for other cloud providers
<a name="concept"></a> <a name="concept"></a>
# Technical Concept # Technical Concept
@ -329,16 +330,17 @@ The installer will start automatically and guide you through the install process
<a name="cloud"></a> <a name="cloud"></a>
## Cloud Deployments ## Cloud Deployments
Located in the [`cloud`](cloud) folder. Located in the [`cloud`](cloud) folder.
Currently there is an example with Ansible. Currently there are examples with Ansible & Terraform.
If you would like to contribute you can add other cloud deployments like Terraform, Chef or Puppet. If you would like to contribute, you can add other cloud deployments like Chef or Puppet or extend current methods with other cloud providers.
<a name="ansible-otc"></a> <a name="ansible"></a>
### Ansible Deployment on Open Telekom Cloud ### Ansible Deployment
You can find an Ansible Playbook based automated T-Pot Deployment in the [`cloud/open-telekom-cloud`](cloud/open-telekom-cloud) folder. You can find an [Ansible](https://www.ansible.com/) based T-Pot deployment in the [`cloud/ansible`](cloud/ansible) folder.
The Playbooks in the [`cloud/open-telekom-cloud/ansible`](cloud/open-telekom-cloud/ansible) folder are reusable across all cloud providers (like AWS, Azure, Digital Ocean). The Playbook in the [`cloud/ansible/openstack`](cloud/ansible/openstack) folder is reusable for all OpenStack clouds out of the box.
The [`deploy_ansible_otc_t-pot.sh`](cloud/open-telekom-cloud/deploy_ansible_otc_t-pot.sh) script is an example of how it works with our own Public Cloud Offering [Open Telekom Cloud](https://open-telekom-cloud.com/en).
It first creates a new Elastic Cloud Server via the Open Telekom Cloud API and then invokes the Ansible Playbooks to install and configure T-Pot. It first creates a new server and then installs and configures T-Pot.
You can have a look at the script and easily adapt it for other cloud providers.
You can have a look at the Playbook and easily adapt the deploy role for other [cloud providers](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html).
<a name="terraform"></a> <a name="terraform"></a>
### Terraform Configuration ### Terraform Configuration

2
cloud/ansible/.gitignore vendored Normal file
View file

@ -0,0 +1,2 @@
# Ansible
*.retry

253
cloud/ansible/README.md Normal file
View file

@ -0,0 +1,253 @@
# T-Pot Ansible
Here you can find a ready-to-use solution for your automated T-Pot deployment using [Ansible](https://www.ansible.com/).
It consists of an Ansible Playbook with multiple roles, which is reusable for all [OpenStack](https://www.openstack.org/) based clouds (e.g. Open Telekom Cloud, Orange Cloud, Telefonica Open Cloud, OVH) out of the box.
Apart from that you can easily adapt the deploy role to use other [cloud providers](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html) (e.g. AWS, Azure, Digital Ocean, Google).
The Playbook first creates a new server and then installs and configures T-Pot.
This example showcases the deployment on our own OpenStack based Public Cloud Offering [Open Telekom Cloud](https://open-telekom-cloud.com/en).
# Table of contents
- [Preparation of Ansible Master](#ansible-master)
- [Ansible Installation](#ansible)
- [Agent Forwarding](#agent-forwarding)
- [Preparations in Open Telekom Cloud Console](#preparation)
- [Create new project](#project)
- [Create API user](#api-user)
- [Import Key Pair](#key-pair)
- [Create VPC, Subnet and Security Group](#vpc-subnet-securitygroup)
- [Clone Git Repository](#clone-git)
- [Settings and recommended values](#settings)
- [OpenStack authentication variables](#os-auth)
- [Ansible remote user](#remote-user)
- [Instance settings](#instance-settings)
- [User password](#user-password)
- [Configure `tpot.conf.dist`](#tpot-conf)
- [Optional: Custom `ews.cfg`](#ews-cfg)
- [Optional: Custom HPFEEDS](#hpfeeds)
- [Deploying a T-Pot](#deploy)
- [Further documentation](#documentation)
<a name="ansible-master"></a>
# Preparation of Ansible Master
You can either run the Ansible Playbook locally on your Linux or macOS machine or you can use an ECS (Elastic Cloud Server) on Open Telekom Cloud, which I did.
I used Ubuntu 18.04 for my Ansible Master Server, but other OSes are fine too.
Ansible works over the SSH Port, so you don't have to add any special rules to your Security Group.
<a name="ansible"></a>
## Ansible Installation
Example for Ubuntu 18.04:
At first we need to add the repository and install Ansible:
`sudo apt-add-repository --yes --update ppa:ansible/ansible`
`sudo apt install ansible`
For other OSes and Distros have a look at the official [Ansible Documentation](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html).
<a name="agent-forwarding"></a>
## Agent Forwarding
Agent Forwarding must be enabled in order to let Ansible do its work.
- On Linux or macOS:
- Create or edit `~/.ssh/config`
- If you run the Ansible Playbook remotely on your Ansible Master Server:
```
Host ANSIBLE_MASTER_IP
ForwardAgent yes
```
- If you run the Ansible Playbook locally, enable it for all hosts, as this includes newly generated T-Pots:
```
Host *
ForwardAgent yes
```
- On Windows using Putty for connecting to your Ansible Master Server:
![Putty Agent Forwarding](doc/putty_agent_forwarding.png)
<a name="preparation"></a>
# Preparations in Open Telekom Cloud Console
(You can skip this if you have already set up an API account, VPC, Subnet and Security Group)
(Just make sure you know the naming for everything, as you will need it to configure the Ansible variables.)
Before we can start deploying, we have to prepare the Open Telekom Cloud tenant.
For that, go to the [Web Console](https://auth.otc.t-systems.com/authui/login) and log in with an admin user.
<a name="project"></a>
## Create new project
I strongly advise you to create a separate project for the T-Pots in your tenant.
In my case I named it `tpot`.
![Create new project](doc/otc_1_project.gif)
<a name="api-user"></a>
## Create API user
The next step is to create a new user account, which is restricted to the project.
This ensures that the API access is limited to that project.
![Create API user](doc/otc_2_user.gif)
<a name="key-pair"></a>
## Import Key Pair
:warning: Now log in with the newly created API user account and select your project.
![Login as API user](doc/otc_3_login.gif)
Import your SSH public key.
![Import SSH Public Key](doc/otc_4_import_key.gif)
<a name="vpc-subnet-securitygroup"></a>
## Create VPC, Subnet and Security Group
- VPC (Virtual Private Cloud) and Subnet:
![Create VPC and Subnet](doc/otc_5_vpc_subnet.gif)
- Security Group:
The configured Security Group should allow all incoming TCP / UDP traffic.
If you want to secure the management interfaces, you can limit the incoming "allow all" traffic to the port range of 1-64000 and allow access to ports > 64000 only from your trusted IPs.
![Create Security Group](doc/otc_6_sec_group.gif)
<a name="clone-git"></a>
# Clone Git Repository
Clone the `tpotce` repository to your Ansible Master:
`git clone https://github.com/dtag-dev-sec/tpotce.git`
All Ansible related files are located in the [`cloud/ansible/openstack`](../../cloud/ansible/openstack) folder.
<a name="settings"></a>
# Settings and recommended values
You can configure all aspects of your Elastic Cloud Server and T-Pot before using the Playbook.
The settings are located in the following Ansible vars files:
<a name="os-auth"></a>
## OpenStack authentication variables
Located at [`openstack/roles/deploy/vars/os_auth.yaml`](openstack/roles/deploy/vars/os_auth.yaml).
Enter your Open Telekom Cloud API user credentials here (username, password, project name, user domain name):
```
auth_url: https://iam.eu-de.otc.t-systems.com/v3
username: your_api_user
password: your_password
project_name: eu-de_your_project
os_user_domain_name: OTC-EU-DE-000000000010000XXXXX
```
You can also perform different authentication methods like sourcing your `.ostackrc` file or using the OpenStack `clouds.yaml` file.
For more information have a look in the [os_server](https://docs.ansible.com/ansible/latest/modules/os_server_module.html) Ansible module documentation.
<a name="remote-user"></a>
## Ansible remote user
You may have to adjust the `remote_user` in the Ansible Playbook under [`openstack/deploy_tpot.yaml`](openstack/deploy_tpot.yaml) depending on your Debian base image (e.g. on Open Telekom Cloud the default Debian user is `linux`).
<a name="instance-settings"></a>
## Instance settings
Located at [`openstack/roles/deploy/vars/main.yaml`](openstack/roles/deploy/vars/main.yaml).
Here you can customize your virtual machine specifications:
- Specify the region name
- Choose an availability zone. For Open Telekom Cloud reference see [here](https://docs.otc.t-systems.com/en-us/endpoint/index.html).
- Change the OS image (For T-Pot we need Debian 9)
- (Optional) Change the volume size
- Specify your key pair
- (Optional) Change the instance type (flavor)
`s2.medium.8` corresponds to 1 vCPU and 8GB of RAM and is the minimum required flavor.
A full list of Open telekom Cloud flavors can be found [here](https://docs.otc.t-systems.com/en-us/usermanual/ecs/en-us_topic_0035470096.html).
- Specify the security group
- Specify the network ID (For Open Telekom Cloud you can find the ID in the Web Console under `Virtual Private Cloud --> your-vpc --> your-subnet --> Network ID`; In general for OpenStack clouds you can use the `python-openstackclient` to retrieve information about your resources)
```
region_name: eu-de
availability_zone: eu-de-03
image: Standard_Debian_9_latest
volume_size: 128
key_name: your-KeyPair
flavor: s2.medium.8
security_groups: your-sg
network: your-network-id
```
<a name="user-password"></a>
## User password
Located at [`openstack/roles/install/vars/main.yaml`](openstack/roles/install/vars/main.yaml).
Here you can set the password for your Debian user (**you should definitely change that**).
```
user_password: LiNuXuSeRPaSs#
```
<a name="tpot-conf"></a>
## Configure `tpot.conf.dist`
The file is located in [`iso/installer/tpot.conf.dist`](../../iso/installer/tpot.conf.dist).
Here you can choose:
- between the various T-Pot editions
- a username for the web interface
- a password for the web interface (**you should definitely change that**)
```
# tpot configuration file
# myCONF_TPOT_FLAVOR=[STANDARD, SENSOR, INDUSTRIAL, COLLECTOR, NEXTGEN]
myCONF_TPOT_FLAVOR='STANDARD'
myCONF_WEB_USER='webuser'
myCONF_WEB_PW='w3b$ecret'
```
<a name="ews-cfg"></a>
## Optional: Custom `ews.cfg`
Enable this by uncommenting the role in the [deploy_tpot.yaml](openstack/deploy_tpot.yaml) playbook.
```
# - custom_ews
```
You can use a custom config file for `ewsposter`.
e.g. when you have your own credentials for delivering data to our [Sicherheitstacho](https://sicherheitstacho.eu/start/main).
You can find the `ews.cfg` template file here: [`openstack/roles/custom_ews/templates/ews.cfg`](openstack/roles/custom_ews/templates/ews.cfg) and adapt it for your needs.
For setting custom credentials, these settings would be relevant for you (the rest of the file can stay as is):
```
[MAIN]
...
contact = your_email_address
...
[EWS]
...
username = your_username
token = your_token
...
```
<a name="hpfeeds"></a>
## Optional: Custom HPFEEDS
Enable this by uncommenting the role in the [deploy_tpot.yaml](openstack/deploy_tpot.yaml) playbook.
```
# - custom_hpfeeds
```
You can specify custom HPFEEDS in [`openstack/roles/custom_hpfeeds/templates/hpfeeds.cfg`](openstack/roles/custom_hpfeeds/templates/hpfeeds.cfg).
That file contains the defaults (turned off) and you can adapt it for your needs, e.g. for SISSDEN:
```
myENABLE=true
myHOST=hpfeeds.sissden.eu
myPORT=10000
myCHANNEL=t-pot.events
myCERT=/opt/ewsposter/sissden.pem
myIDENT=your_user
mySECRET=your_secret
myFORMAT=json
```
<a name="deploy"></a>
# Deploying a T-Pot :honey_pot::honeybee:
Now, after configuring everything, we can finally start deploying T-Pots!
Go to the [`openstack`](openstack) folder and run the Ansible Playbook with:
`ansible-playbook deploy_tpot.yaml`
(Yes, it is as easy as that :smile:)
If you are running on a machine which asks for a sudo password, you can use:
`ansible-playbook --ask-become-pass deploy_tpot.yaml`
The Playbook will first install required packages on the Ansible Master and then deploy a new server instance.
After that, T-Pot gets installed and configured on the newly created host, optionally custom configs are applied and finally it reboots.
<a name="documentation"></a>
# Further documentation
- [Ansible Documentation](https://docs.ansible.com/ansible/latest/)
- [Cloud modules — Ansible Documentation](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html)
- [os_server – Create/Delete Compute Instances from OpenStack — Ansible Documentation](https://docs.ansible.com/ansible/latest/modules/os_server_module.html)
- [Open Telekom Cloud Help Center](https://docs.otc.t-systems.com/)
- [Open Telekom Cloud API Overview](https://docs.otc.t-systems.com/en-us/api/wp/en-us_topic_0052070394.html)

0
cloud/open-telekom-cloud/doc/otc_1_project.gif → cloud/ansible/doc/otc_1_project.gif
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 204 KiB

After

Width:  |  Height:  |  Size: 204 KiB

0
cloud/open-telekom-cloud/doc/otc_2_user.gif → cloud/ansible/doc/otc_2_user.gif
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 883 KiB

After

Width:  |  Height:  |  Size: 883 KiB

0
cloud/open-telekom-cloud/doc/otc_3_login.gif → cloud/ansible/doc/otc_3_login.gif
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 148 KiB

After

Width:  |  Height:  |  Size: 148 KiB

0
cloud/open-telekom-cloud/doc/otc_4_import_key.gif → cloud/ansible/doc/otc_4_import_key.gif
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 193 KiB

After

Width:  |  Height:  |  Size: 193 KiB

0
cloud/open-telekom-cloud/doc/otc_5_vpc_subnet.gif → cloud/ansible/doc/otc_5_vpc_subnet.gif
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 172 KiB

After

Width:  |  Height:  |  Size: 172 KiB

0
cloud/open-telekom-cloud/doc/otc_6_sec_group.gif → cloud/ansible/doc/otc_6_sec_group.gif
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 337 KiB

After

Width:  |  Height:  |  Size: 337 KiB

0
cloud/open-telekom-cloud/doc/putty_agent_forwarding.png → cloud/ansible/doc/putty_agent_forwarding.png
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 23 KiB

After

Width:  |  Height:  |  Size: 23 KiB

5
cloud/ansible/openstack/ansible.cfg Normal file
View file

@ -0,0 +1,5 @@
[defaults]
host_key_checking = false
[ssh_connection]
scp_if_ssh = true

25
cloud/ansible/openstack/deploy_tpot.yaml Normal file
View file

@ -0,0 +1,25 @@
- name: Check host prerequisites
hosts: localhost
become: yes
become_user: root
become_method: sudo
roles:
- check
- name: Deploy instance
hosts: localhost
roles:
- deploy
- name: Install T-Pot on new instance
hosts: TPOT
remote_user: linux
become: yes
become_user: root
become_method: sudo
gather_facts: no
roles:
- install
# - custom_ews
# - custom_hpfeeds
- reboot

28
cloud/ansible/openstack/roles/check/tasks/main.yaml Normal file
View file

@ -0,0 +1,28 @@
- name: Install pwgen
package:
name: pwgen
state: present
- name: Install setuptools
package:
name: python-setuptools
state: present
- name: Install pip
package:
name: python-pip
state: present
- name: Install openstacksdk
pip:
name: openstacksdk
- name: Set fact for agent forwarding
set_fact:
agent_forwarding: "{{ lookup('env','SSH_AUTH_SOCK') }}"
- name: Check if agent forwarding is enabled
fail:
msg: Please enable agent forwarding to allow Ansible to connect to the remote host!
ignore_errors: yes
when: agent_forwarding == ""

11
cloud/open-telekom-cloud/ansible/roles/custom_ews/tasks/main.yaml → cloud/ansible/openstack/roles/custom_ews/tasks/main.yaml
View file

@ -11,14 +11,3 @@
path: /opt/tpot/etc/tpot.yml path: /opt/tpot/etc/tpot.yml
insertafter: '/opt/ewsposter/ews.ip' insertafter: '/opt/ewsposter/ews.ip'
line: ' - /data/ews/conf/ews.cfg:/opt/ewsposter/ews.cfg' line: ' - /data/ews/conf/ews.cfg:/opt/ewsposter/ews.cfg'
- name: Copy hpfeeds configuration file
template:
src: ../templates/hpfeeds.cfg
dest: /data/ews/conf
owner: root
group: root
mode: 0644
- name: Applying hpfeeds settings
command: /opt/tpot/bin/hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg

30
cloud/open-telekom-cloud/ansible/roles/custom_ews/templates/ews.cfg → cloud/ansible/openstack/roles/custom_ews/templates/ews.cfg
View file

@ -35,7 +35,7 @@ jsondir = /data/ews/json/
[GLASTOPFV3] [GLASTOPFV3]
glastopfv3 = true glastopfv3 = true
nodeid = glastopfv3-{{ HPNAME }} nodeid = glastopfv3-{{ ansible_hostname }}
sqlitedb = /data/glastopf/db/glastopf.db sqlitedb = /data/glastopf/db/glastopf.db
malwaredir = /data/glastopf/data/files/ malwaredir = /data/glastopf/data/files/
@ -59,18 +59,18 @@ malwaredir =
[COWRIE] [COWRIE]
cowrie = true cowrie = true
nodeid = cowrie-{{ HPNAME }} nodeid = cowrie-{{ ansible_hostname }}
logfile = /data/cowrie/log/cowrie.json logfile = /data/cowrie/log/cowrie.json
[DIONAEA] [DIONAEA]
dionaea = true dionaea = true
nodeid = dionaea-{{ HPNAME }} nodeid = dionaea-{{ ansible_hostname }}
malwaredir = /data/dionaea/binaries/ malwaredir = /data/dionaea/binaries/
sqlitedb = /data/dionaea/log/dionaea.sqlite sqlitedb = /data/dionaea/log/dionaea.sqlite
[HONEYTRAP] [HONEYTRAP]
honeytrap = true honeytrap = true
nodeid = honeytrap-{{ HPNAME }} nodeid = honeytrap-{{ ansible_hostname }}
newversion = true newversion = true
payloaddir = /data/honeytrap/attacks/ payloaddir = /data/honeytrap/attacks/
attackerfile = /data/honeytrap/log/attacker.log attackerfile = /data/honeytrap/log/attacker.log
@ -83,55 +83,55 @@ targetip =
[EMOBILITY] [EMOBILITY]
eMobility = false eMobility = false
nodeid = emobility-{{ HPNAME }} nodeid = emobility-{{ ansible_hostname }}
logfile = /data/emobility/log/centralsystemEWS.log logfile = /data/emobility/log/centralsystemEWS.log
[CONPOT] [CONPOT]
conpot = true conpot = true
nodeid = conpot-{{ HPNAME }} nodeid = conpot-{{ ansible_hostname }}
logfile = /data/conpot/log/conpot*.json logfile = /data/conpot/log/conpot*.json
[ELASTICPOT] [ELASTICPOT]
elasticpot = true elasticpot = true
nodeid = elasticpot-{{ HPNAME }} nodeid = elasticpot-{{ ansible_hostname }}
logfile = /data/elasticpot/log/elasticpot.log logfile = /data/elasticpot/log/elasticpot.log
[SURICATA] [SURICATA]
suricata = true suricata = true
nodeid = suricata-{{ HPNAME }} nodeid = suricata-{{ ansible_hostname }}
logfile = /data/suricata/log/eve.json logfile = /data/suricata/log/eve.json
[MAILONEY] [MAILONEY]
mailoney = true mailoney = true
nodeid = mailoney-{{ HPNAME }} nodeid = mailoney-{{ ansible_hostname }}
logfile = /data/mailoney/log/commands.log logfile = /data/mailoney/log/commands.log
[RDPY] [RDPY]
rdpy = true rdpy = true
nodeid = rdpy-{{ HPNAME }} nodeid = rdpy-{{ ansible_hostname }}
logfile = /data/rdpy/log/rdpy.log logfile = /data/rdpy/log/rdpy.log
[VNCLOWPOT] [VNCLOWPOT]
vnclowpot = true vnclowpot = true
nodeid = vnclowpot-{{ HPNAME }} nodeid = vnclowpot-{{ ansible_hostname }}
logfile = /data/vnclowpot/log/vnclowpot.log logfile = /data/vnclowpot/log/vnclowpot.log
[HERALDING] [HERALDING]
heralding = true heralding = true
nodeid = heralding-{{ HPNAME }} nodeid = heralding-{{ ansible_hostname }}
logfile = /data/heralding/log/auth.csv logfile = /data/heralding/log/auth.csv
[CISCOASA] [CISCOASA]
ciscoasa = true ciscoasa = true
nodeid = ciscoasa-{{ HPNAME }} nodeid = ciscoasa-{{ ansible_hostname }}
logfile = /data/ciscoasa/log/ciscoasa.log logfile = /data/ciscoasa/log/ciscoasa.log
[TANNER] [TANNER]
tanner = true tanner = true
nodeid = tanner-{{ HPNAME }} nodeid = tanner-{{ ansible_hostname }}
logfile = /data/tanner/log/tanner_report.json logfile = /data/tanner/log/tanner_report.json
[GLUTTON] [GLUTTON]
glutton = true glutton = true
nodeid = glutton-{{ HPNAME }} nodeid = glutton-{{ ansible_hostname }}
logfile = /data/glutton/log/glutton.log logfile = /data/glutton/log/glutton.log

10
cloud/ansible/openstack/roles/custom_hpfeeds/tasks/main.yaml Normal file
View file

@ -0,0 +1,10 @@
- name: Copy hpfeeds configuration file
template:
src: ../templates/hpfeeds.cfg
dest: /data/ews/conf
owner: root
group: root
mode: 0644
- name: Applying hpfeeds settings
command: /opt/tpot/bin/hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg

0
cloud/open-telekom-cloud/ansible/roles/custom_ews/templates/hpfeeds.cfg → cloud/ansible/openstack/roles/custom_hpfeeds/templates/hpfeeds.cfg
View file

34
cloud/ansible/openstack/roles/deploy/tasks/main.yaml Normal file
View file

@ -0,0 +1,34 @@
- name: Create T-Pot name
shell: echo t-pot-ansible-$(pwgen -ns 6 -1)
register: tpot_name
- name: Import OpenStack authentication variables
include_vars:
file: roles/deploy/vars/os_auth.yaml
- name: Launch an instance
os_server:
auth:
auth_url: "{{ auth_url }}"
username: "{{ username }}"
password: "{{ password }}"
project_name: "{{ project_name }}"
os_user_domain_name: "{{ os_user_domain_name }}"
name: "{{ tpot_name.stdout }}"
region_name: "{{ region_name }}"
availability_zone: "{{ availability_zone }}"
image: "{{ image }}"
boot_from_volume: yes
volume_size: "{{ volume_size }}"
key_name: "{{ key_name }}"
timeout: 200
flavor: "{{ flavor }}"
security_groups: "{{ security_groups }}"
network: "{{ network }}"
register: tpot
- name: Add instance to inventory
add_host:
hostname: "{{ tpot_name.stdout }}"
ansible_host: "{{ tpot.server.public_v4 }}"
groups: TPOT

8
cloud/ansible/openstack/roles/deploy/vars/main.yaml Normal file
View file

@ -0,0 +1,8 @@
region_name: eu-de
availability_zone: eu-de-03
image: Standard_Debian_9_latest
volume_size: 128
key_name: your-KeyPair
flavor: s2.medium.8
security_groups: your-sg
network: your-network-id

5
cloud/ansible/openstack/roles/deploy/vars/os_auth.yaml Normal file
View file

@ -0,0 +1,5 @@
auth_url: https://iam.eu-de.otc.t-systems.com/v3
username: your_api_user
password: your_password
project_name: eu-de_your_project
os_user_domain_name: OTC-EU-DE-000000000010000XXXXX

21
cloud/open-telekom-cloud/ansible/roles/install/tasks/main.yaml → cloud/ansible/openstack/roles/install/tasks/main.yaml
View file

@ -3,28 +3,29 @@
delay: 30 delay: 30
timeout: 300 timeout: 300
- name: Gathering Facts - name: Gathering facts
setup: setup:
- name: Cloning t-pot install directory - name: Cloning T-Pot install directory
git: git:
repo: 'https://github.com/dtag-dev-sec/tpotce.git' repo: "https://github.com/dtag-dev-sec/tpotce.git"
dest: /root/tpot dest: /root/tpot
- name: Prepare to set user password - name: Prepare to set user password
set_fact: set_fact:
user_password: "{{ lookup('env', 'LINUX_PASS') }}" user_name: "{{ ansible_user }}"
user_salt: 's0mew1ck3dTpoT' user_password: "{{ user_password }}"
user_salt: "s0mew1ck3dTpoT"
- name: Changing password for user linux to {{ user_password }} - name: Changing password for user {{ user_name }} to {{ user_password }}
user: user:
name: "linux" name: "{{ ansible_user }}"
password: "{{ user_password | password_hash('sha512', user_salt) }}" password: "{{ user_password | password_hash('sha512', user_salt) }}"
state: present state: present
shell: /bin/bash shell: /bin/bash
update_password: always update_password: always
- name: Copy t-pot configuration file - name: Copy T-Pot configuration file
template: template:
src: ../../../../../../iso/installer/tpot.conf.dist src: ../../../../../../iso/installer/tpot.conf.dist
dest: /root/tpot.conf dest: /root/tpot.conf
@ -32,10 +33,10 @@
group: root group: root
mode: 0644 mode: 0644
- name: Install t-pot on ECS - be patient, this might take 15 to 30 minutes depending on the connection speed. No further output is given. - name: Install T-Pot on instance - be patient, this might take 15 to 30 minutes depending on the connection speed. No further output is given.
command: /root/tpot/iso/installer/install.sh --type=auto --conf=/root/tpot.conf command: /root/tpot/iso/installer/install.sh --type=auto --conf=/root/tpot.conf
- name: Delete t-pot configuration file - name: Delete T-Pot configuration file
file: file:
path: /root/tpot.conf path: /root/tpot.conf
state: absent state: absent

1
cloud/ansible/openstack/roles/install/vars/main.yaml Normal file
View file

@ -0,0 +1 @@
user_password: LiNuXuSeRPaSs#

15
cloud/ansible/openstack/roles/reboot/tasks/main.yaml Normal file
View file

@ -0,0 +1,15 @@
- name: Finally rebooting T-Pot in one minute
shell: /sbin/shutdown -r -t 1
become: true
- name: Next login options
debug:
msg:
- "***** SSH Access:"
- "***** ssh {{ ansible_user }}@{{ ansible_host }} -p 64295"
- ""
- "***** Web UI:"
- "***** https://{{ ansible_host }}:64297"
- ""
- "***** Admin UI:"
- "***** https://{{ ansible_host }}:64294"

15
cloud/open-telekom-cloud/.ecs_settings.sh
View file

@ -1,15 +0,0 @@
# Set password for user linux
linuxpass=LiNuXuSeRPaSs#
# Custom EWS config
custom_ews=false
# Set ECS related stuff
instance=s2.medium.8
imagename=Standard_Debian_9_latest
subnet=your-subnet
vpcname=your-vpc
secgroup=your-sg
keyname=your-KeyPair
disksize=128
az=eu-de-03

11
cloud/open-telekom-cloud/.gitignore vendored
View file

@ -1,11 +0,0 @@
# Ansible
*.retry
# Generated hosts
hosts/
# Cloned git repository
otc-tools/
# All log files
*.log

5
cloud/open-telekom-cloud/.otc_env.sh
View file

@ -1,5 +0,0 @@
export OS_USERNAME=your_api_user
export OS_PASSWORD=your_password
export OS_USER_DOMAIN_NAME=OTC-EU-DE-000000000010000XXXXX
export OS_PROJECT_NAME=eu-de_your_project
export OS_AUTH_URL=https://iam.eu-de.otc.t-systems.com/v3

228
cloud/open-telekom-cloud/README.md
View file

@ -1,228 +0,0 @@
# Ansible T-Pot Deployment on Open Telekom Cloud :cloud:
Here you can find a ready-to-use solution for your automated T-Pot deployment using [Ansible](https://www.ansible.com/).
It consists of multiple Ansible Playbooks, which can be reused across all Cloud Providers (like AWS, Azure, Digital Ocean).
This example showcases the deployment on our own Public Cloud Offering [Open Telekom Cloud](https://open-telekom-cloud.com/en).
# Table of contents
- [Installation of Ansible Master](#installation)
- [Packages](#packages)
- [Agent Forwarding](#agent-forwarding)
- [Preparations in Open Telekom Cloud Console](#preparation)
- [Create new project](#project)
- [Create API user](#api-user)
- [Import Key Pair](#key-pair)
- [Create VPC, Subnet and Security Group](#vpc-subnet-securitygroup)
- [Clone Git Repository](#clone-git)
- [Settings and recommended values](#settings)
- [Configure `.otc_env.sh`](#otc-env)
- [Configure `.ecs_settings.sh`](#ecs-settings)
- [Configure `tpot.conf.dist`](#tpot-conf)
- [Optional: Custom `ews.cfg` and HPFEEDS](#ews-hpfeeds)
- [Deploying a T-Pot](#deploy)
- [Further documentation](#documentation)
<a name="installation"></a>
# Installation of Ansible Master
You can either run the deploy script locally on your Linux or MacOS machine or you can use an ECS (Elastic Cloud Server) on Open Telekom Cloud, which I did.
I used Ubuntu 18.04 for my Ansible Master Server, but other OSes are fine too.
Ansible works over the SSH Port, so you don't have to add any special rules to you Security Group.
<a name="packages"></a>
## Packages
At first we need to add the repository and install Ansible:
`sudo apt-add-repository --yes --update ppa:ansible/ansible`
`sudo apt install ansible`
Also we need **pwegen** (for creating T-Pot names) and **jq** (a JSON processor):
`sudo apt install pwgen jq`
<a name="agent-forwarding"></a>
## Agent Forwarding
Agent forwarding must be enabled in order to let Ansible do its work.
- On Linux or MacOS:
- Create or edit `~/.ssh/config`
- If you execute the script remotely on your Ansible Master Server:
```
Host ANSIBLE_MASTER_IP
ForwardAgent yes
```
- If you execute the script locally, enable it for all Hosts, as this includes newly generated T-Pots:
```
Host *
ForwardAgent yes
```
- On Windows using Putty:
![Putty Agent Forwarding](doc/putty_agent_forwarding.png)
<a name="preparation"></a>
# Preparations in Open Telekom Cloud Console
(You can skip this if you have already set up an API account, VPC and ...)
(Just make sure you know the naming for everything, as you will need it to configure the script.)
Before we can start deploying, we have to prepare the Open Telekom Cloud Tennant.
For that, go to the [Web Console](https://auth.otc.t-systems.com/authui/login) and log in with an admin user.
<a name="project"></a>
## Create new project
I strongly advise you, to create a separate project for the T-Pots in your tennant.
In my case I named it `tpot`.
![Create new project](doc/otc_1_project.gif)
<a name="api-user"></a>
## Create API user
The next step is to create a new user account, which is restricted to the project.
This ensures that the API access is limited to that project.
![Create API user](doc/otc_2_user.gif)
<a name="key-pair"></a>
## Import Key Pair
:warning: Now log in with the newly created user account and select your project.
![Login as API user](doc/otc_3_login.gif)
Import your SSH public key.
![Import SSH Public Key](doc/otc_4_import_key.gif)
<a name="vpc-subnet-securitygroup"></a>
## Create VPC, Subnet and Security Group
- VPC (Virtual Private Cloud) and Subnet:
![Create VPC and Subnet](doc/otc_5_vpc_subnet.gif)
- Security Group:
The configured Security Group should allow all incoming TCP / UDP traffic.
If you want to secure the management interfaces, you can limit the incoming "allow all" traffic to the port range of 1-64000 and allow access to ports > 64000 only from your trusted IPs.
![Create Security Group](doc/otc_6_sec_group.gif)
<a name="clone-git"></a>
# Clone Git Repository
Clone the `tpotce` repository to your Ansible Master:
`git clone https://github.com/dtag-dev-sec/tpotce.git`
All Ansible and automatic deployment related files are located in the [`cloud/open-telekom-cloud`](../../cloud/open-telekom-cloud) folder.
<a name="settings"></a>
# Settings and recommended values
You can configure all aspects of your ECS and T-Pot before using the script.
The settings are located in the following files:
<a name="otc-env"></a>
## Configure `.otc_env.sh`
Enter your Open Telekom Cloud API user credentials here (username, password, tennant-ID, project name):
```
export OS_USERNAME=your_api_user
export OS_PASSWORD=your_password
export OS_USER_DOMAIN_NAME=OTC-EU-DE-000000000010000XXXXX
export OS_PROJECT_NAME=eu-de_your_project
export OS_AUTH_URL=https://iam.eu-de.otc.t-systems.com/v3
```
<a name="ecs-settings"></a>
## Configure `.ecs_settings.sh`
Here you can customize your Elastic Cloud Server (ECS):
- Password for the user `linux` (**you should definitely change that**)
You may have to adjust the `remote_user` in the Ansible Playbooks under [ansible](ansible) if you are using a normal/default Debian base image
- (Optional) For using a custom `ews.cfg` set to `true`; See here: [Optional: Custom `ews.cfg`](#ews-cfg)
- (Optional) Change the instance type (flavor) of the ECS.
`s2.medium.8` corresponds to 1 vCPU and 8GB of RAM and is the minimum required flavor.
A full list of flavors can be found [here](https://docs.otc.t-systems.com/en-us/usermanual/ecs/en-us_topic_0035470096.html).
- Change the OS (Don't touch; for T-Pot we need Debian 9)
- Specify the VPC, Subnet, Security Group and Key Pair you created before
- (Optional) Change the disk size
- You can choose from multiple Availibility Zones (AZ). For reference see [here](https://docs.otc.t-systems.com/en-us/endpoint/index.html).
```
# Set password for user linux
linuxpass=LiNuXuSeRPaSs#
# Custom EWS config
custom_ews=false
# Set ECS related stuff
instance=s2.medium.8
imagename=Standard_Debian_9_latest
subnet=your-subnet
vpcname=your-vpc
secgroup=your-sg
keyname=your-KeyPair
disksize=128
az=eu-de-03
```
<a name="tpot-conf"></a>
## Configure `tpot.conf.dist`
The file is located in [`iso/installer/tpot.conf.dist`](../../iso/installer/tpot.conf.dist).
Here you can choose:
- between the various T-Pot editions
- a username for the web interface
- a password for the web interface (**you should definitely change that**)
```
# tpot configuration file
# myCONF_TPOT_FLAVOR=[STANDARD, SENSOR, INDUSTRIAL, COLLECTOR, NEXTGEN, LEGACY]
myCONF_TPOT_FLAVOR='STANDARD'
myCONF_WEB_USER='webuser'
myCONF_WEB_PW='w3b$ecret'
```
<a name="ews-hpfeeds"></a>
## Optional: Custom `ews.cfg` and HPFEEDS
To enable these features, set `custom_ews=true` in `.ecs_settings.sh`; See here: [Configure `.ecs_settings.sh`](#ecs-settings)
### ews.cfg
You can use a custom config file for `ewsposter`.
e.g. when you have your own credentials for delivering data to our [Sicherheitstacho](https://sicherheitstacho.eu/start/main).
You can find the `ews.cfg` template file here: [`ansible/roles/custom_ews/templates/ews.cfg`](ansible/roles/custom_ews/templates/ews.cfg) and adapt it for your needs.
For setting custom credentials, these settings would be relevant for you (the rest of the file can stay as is):
```
[MAIN]
...
contact = your_email_address
...
[EWS]
...
username = your_username
token = your_token
...
```
### HPFEEDS
You can also specify HPFEEDS in [`ansible/roles/custom_ews/templates/hpfeeds.cfg`](ansible/roles/custom_ews/templates/hpfeeds.cfg).
That file constains the defaults (turned off) and you can adapt it for your needs, e.g. for SISSDEN:
```
myENABLE=true
myHOST=hpfeeds.sissden.eu
myPORT=10000
myCHANNEL=t-pot.events
myCERT=/opt/ewsposter/sissden.pem
myIDENT=your_user
mySECRET=your_secret
myFORMAT=json
```
<a name="deploy"></a>
# Deploying a T-Pot :honey_pot::honeybee:
Now, after configuring everything, we can finally start deploying T-Pots:
`./deploy_ansible_otc_t-pot.sh`
(Yes, it is as easy as that :smile:)
The script will first create an Open Telekom Cloud ECS via the API.
After that, the Ansible Playbooks are executed on the newly created Host to install the T-Pot and configure everything.
You can see the progress of every step in the console output.
If something should go wrong, you will be provided with an according error message, that you can hopefully act upon and retry.
<a name="documentation"></a>
# Further documentation
- [Ansible Documentation](https://docs.ansible.com/ansible/latest/)
- [Open Telekom Cloud Help Center](https://docs.otc.t-systems.com/)
- [Open Telekom Cloud API Overview](https://docs.otc.t-systems.com/en-us/api/wp/en-us_topic_0052070394.html)
- [otc-tools](https://github.com/OpenTelekomCloud/otc-tools) on GitHub

10
cloud/open-telekom-cloud/ansible/custom_ews.yaml
View file

@ -1,10 +0,0 @@
# This playbook sets a custom EWS config on the T-Pot
- hosts: TPOT
remote_user: linux
become: yes
become_user: root
become_method: sudo
roles:
- custom_ews

13
cloud/open-telekom-cloud/ansible/install.yaml
View file

@ -1,13 +0,0 @@
# This playbook deploys a T-Pot
- hosts: TPOT
remote_user: linux
become: yes
become_user: root
become_method: sudo
gather_facts: no
roles:
- install

12
cloud/open-telekom-cloud/ansible/reboot.yaml
View file

@ -1,12 +0,0 @@
# This playbook reboots a T-Pot
- hosts: TPOT
remote_user: linux
become: yes
become_user: root
become_method: sudo
tasks:
- name: Finally rebooting t-pot in one minute - make sure your next login is on port 64295 or via https:// on port 64297
shell: /sbin/shutdown -r -t 1
become: true

122
cloud/open-telekom-cloud/deploy_ansible_otc_t-pot.sh
View file

@ -1,122 +0,0 @@
#!/bin/bash
# Check if required packages are installed
if ! hash ansible 2>/dev/null; then
echo "### Package 'ansible' is missing. Please install it with:"
echo " sudo apt-add-repository --yes --update ppa:ansible/ansible"
echo " sudo apt install ansible"
exit 1
fi
if ! hash pwgen 2>/dev/null; then
echo "### Package 'pwgen' is missing. Please install it with:"
echo " sudo apt install pwgen"
exit 1
fi
if ! hash jq 2>/dev/null; then
echo "### Package 'jq' is missing. Please install it with:"
echo " sudo apt install jq"
exit 1
fi
# Check for Agent Forwarding
if ! printenv | grep SSH_AUTH_SOCK > /dev/null; then
echo "### Agent forwarding seems to be disabled."
echo "### In order to let Ansible do its work, please enable it."
exit 1
fi
# Import ECS settings
source .ecs_settings.sh
# Import OTC authentication credentials
source .otc_env.sh
# Password is later used by Ansible
export LINUX_PASS=$linuxpass
# Ignore ssh host keys as they are new anyway
export ANSIBLE_HOST_KEY_CHECKING=False
# Create hosts directory
mkdir -p hosts
# Create random ID
HPNAME=t-pot-otc-$(pwgen -ns 6 -1)
# Get otc-tools
echo "### Cloning otc-tools..."
git clone https://github.com/OpenTelekomCloud/otc-tools.git 2>/dev/null
# Create ECS via OTC API
echo "### Creating new ECS host via OTC API..."
./otc-tools/otc.sh ecs create \
--instance-type $instance\
--instance-name $HPNAME\
--image-name $imagename\
--subnet-name $subnet\
--vpc-name $vpcname\
--security-group-name $secgroup\
--admin-pass $linuxpass\
--key-name $keyname\
--public true\
--disksize $disksize\
--disktype SATA\
--az $az\
--wait \
2> otc_tools.log
if [ $? -eq 0 ]; then
if [ "$(uname)" == "Darwin" ]; then
PUBIP=$(./otc-tools/otc.sh ecs list 2>/dev/null | grep $HPNAME|cut -d "," -f2 |cut -d "\"" -f 2)
else
PUBIP=$(./otc-tools/otc.sh ecs list 2>/dev/null | grep $HPNAME|cut -d " " -f17)
fi
echo "[TPOT]" > ./hosts/$HPNAME
echo $PUBIP HPNAME=$HPNAME>> ./hosts/$HPNAME
echo "### NEW HOST $HPNAME ON IP $PUBIP"
ansible-playbook -i ./hosts/$HPNAME ./ansible/install.yaml
if [ $custom_ews = true ]; then
ansible-playbook -i ./hosts/$HPNAME ./ansible/custom_ews.yaml
fi
ansible-playbook -i ./hosts/$HPNAME ./ansible/reboot.yaml
echo "***********************************************"
echo "***** SSH TO TARGET: "
echo "***** ssh linux@$PUBIP -p 64295"
echo "***********************************************"
else
if grep '401 Unauthorized' otc_tools.log > /dev/null; then
echo "### API username or password is incorrect"
elif grep 'Flavor' otc_tools.log > /dev/null; then
echo "### Specified ECS Flavor not found"
elif grep 'No image found by name' otc_tools.log > /dev/null; then
echo "### Specified Image not found"
elif grep 'No subnet found by name' otc_tools.log > /dev/null; then
echo "### Specified Subnet not found"
elif grep 'No VPC found by name' otc_tools.log > /dev/null; then
echo "### Specified VPC not found"
elif grep 'No security-group found by name' otc_tools.log > /dev/null; then
echo "### Specified Security Group not found"
elif grep 'Invalid key_name provided' otc_tools.log > /dev/null; then
echo "### Specified Key Pair not found"
elif grep 'availability_zone' otc_tools.log > /dev/null; then
echo "### Specified Availability Zone not found"
elif grep 'quota' otc_tools.log > /dev/null; then
echo "### Quota exceeded. Please check your available quotas online"
echo "### You can either delete unused resources or apply for a higher quota"
fi
echo "### ECS creation unsuccessful. Aborting..."
fi