Merge pull request #769 from shaderecker/ansible

Ansible updates
This commit is contained in:
Marco Ochse 2021-02-15 10:12:14 +01:00 committed by GitHub
commit b01bf50aaf
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
18 changed files with 142 additions and 105 deletions

View file

@ -290,9 +290,9 @@ If you would like to contribute, you can add other cloud deployments like Chef o
You can find an [Ansible](https://www.ansible.com/) based T-Pot deployment in the [`cloud/ansible`](cloud/ansible) folder.
The Playbook in the [`cloud/ansible/openstack`](cloud/ansible/openstack) folder is reusable for all **OpenStack** clouds out of the box.
It first creates all resources (security group, network, subnet, router), deploys a new server and then installs and configures T-Pot.
It first creates all resources (security group, network, subnet, router), deploys one (or more) new servers and then installs and configures T-Pot on them.
You can have a look at the Playbook and easily adapt the deploy role for other [cloud providers](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html).
You can have a look at the Playbook and easily adapt the deploy role for other [cloud providers](https://docs.ansible.com/ansible/latest/scenario_guides/cloud_guides.html). Check out [Ansible Galaxy](https://galaxy.ansible.com/search?keywords=&order_by=-relevance&page=1&deprecated=false&type=collection&tags=cloud) for more cloud collections.
*Please note*: Cloud providers usually offer adjusted Debian OS images, which might not be compatible with T-Pot. There is no cloud provider support provided of any kind.
@ -304,7 +304,7 @@ You can find [Terraform](https://www.terraform.io/) configuration in the [`cloud
This can be used to launch a virtual machine, bootstrap any dependencies and install T-Pot in a single step.
Configuration for **Amazon Web Services** (AWS) and **Open Telekom Cloud** (OTC) is currently included.
This can easily be extended to support other [Terraform providers](https://www.terraform.io/docs/providers/index.html).
This can easily be extended to support other [Terraform providers](https://registry.terraform.io/browse/providers?category=public-cloud%2Ccloud-automation%2Cinfrastructure).
*Please note*: Cloud providers usually offer adjusted Debian OS images, which might not be compatible with T-Pot. There is no cloud provider support provided of any kind.

10
cloud/.gitignore vendored Normal file
View file

@ -0,0 +1,10 @@
# Ansible
*.retry
# Terraform
**/.terraform
**/terraform.*
# OpenStack clouds
clouds.yaml
secure.yaml

View file

@ -1,2 +0,0 @@
# Ansible
*.retry

View file

@ -2,15 +2,16 @@
Here you can find a ready-to-use solution for your automated T-Pot deployment using [Ansible](https://www.ansible.com/).
It consists of an Ansible Playbook with multiple roles, which is reusable for all [OpenStack](https://www.openstack.org/) based clouds (e.g. Open Telekom Cloud, Orange Cloud, Telefonica Open Cloud, OVH) out of the box.
Apart from that you can easily adapt the deploy role to use other [cloud providers](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html) (e.g. AWS, Azure, Digital Ocean, Google).
Apart from that you can easily adapt the deploy role to use other [cloud providers](https://docs.ansible.com/ansible/latest/scenario_guides/cloud_guides.html). Check out [Ansible Galaxy](https://galaxy.ansible.com/search?keywords=&order_by=-relevance&page=1&deprecated=false&type=collection&tags=cloud) for more cloud collections.
The Playbook first creates all resources (security group, network, subnet, router), deploys a new server and then installs and configures T-Pot.
The Playbook first creates all resources (security group, network, subnet, router), deploys one (or more) new servers and then installs and configures T-Pot on them.
This example showcases the deployment on our own OpenStack based Public Cloud Offering [Open Telekom Cloud](https://open-telekom-cloud.com/en).
# Table of contents
- [Preparation of Ansible Master](#ansible-master)
- [Ansible Installation](#ansible)
- [OpenStack Collection Installation](#collection)
- [Agent Forwarding](#agent-forwarding)
- [Preparations in Open Telekom Cloud Console](#preparation)
- [Create new project](#project)
@ -18,8 +19,9 @@ This example showcases the deployment on our own OpenStack based Public Cloud Of
- [Import Key Pair](#key-pair)
- [Clone Git Repository](#clone-git)
- [Settings and recommended values](#settings)
- [Clouds.yaml](#clouds-yaml)
- [clouds.yaml](#clouds-yaml)
- [Ansible remote user](#remote-user)
- [Number of instances to deploy](#number)
- [Instance settings](#instance-settings)
- [User password](#user-password)
- [Configure `tpot.conf.dist`](#tpot-conf)
@ -56,6 +58,11 @@ In short (if you already have Python3/pip3 installed):
pip3 install ansible
```
<a name="collection"></a>
## OpenStack Collection Installation
For interacting with OpenStack resources in Ansible, you need to install the collection from Ansible Galaxy:
`ansible-galaxy collection install openstack.cloud`
<a name="agent-forwarding"></a>
## Agent Forwarding
If you run the Ansible Playbook remotely on your Ansible Master Server, Agent Forwarding must be enabled in order to let Ansible connect to newly created machines.
@ -112,7 +119,7 @@ All Ansible related files are located in the [`cloud/ansible/openstack`](opensta
You can configure all aspects of your Elastic Cloud Server and T-Pot before using the Playbook:
<a name="clouds-yaml"></a>
## Clouds.yaml
## clouds.yaml
Located at [`openstack/clouds.yaml`](openstack/clouds.yaml).
Enter your Open Telekom Cloud API user credentials here (username, password, project name, user domain name):
```
@ -126,22 +133,36 @@ clouds:
user_domain_name: OTC-EU-DE-000000000010000XXXXX
```
You can also perform different authentication methods like sourcing OpenStack OS_* environment variables or providing an inline dictionary.
For more information have a look in the [os_server](https://docs.ansible.com/ansible/latest/modules/os_server_module.html) Ansible module documentation.
For more information have a look in the [openstack.cloud.server](https://docs.ansible.com/ansible/latest/collections/openstack/cloud/server_module.html) Ansible module documentation.
If you already have your own `clouds.yaml` file or have multiple clouds in there, you can specify which one to use in the `openstack/my_os_cloud.yaml` file:
```
# Enter the name of your cloud to use from clouds.yaml
cloud: open-telekom-cloud
```
<a name="remote-user"></a>
## Ansible remote user
You may have to adjust the `remote_user` in the Ansible Playbook under [`openstack/deploy_tpot.yaml`](openstack/deploy_tpot.yaml) depending on your Debian base image (e.g. on Open Telekom Cloud the default Debian user is `linux`).
<a name="number"></a>
## Number of instances to deploy
You can adjust the number of VMs/T-Pots that you want to create in [`openstack/deploy_tpot.yaml`](openstack/deploy_tpot.yaml):
```
loop: "{{ range(0, 1) }}"
```
One instance is set as the default, increase to your liking.
<a name="instance-settings"></a>
## Instance settings
Located at [`openstack/roles/deploy/vars/main.yaml`](openstack/roles/deploy/vars/main.yaml).
Located at [`openstack/roles/create_vm/vars/main.yaml`](openstack/roles/create_vm/vars/main.yaml).
Here you can customize your virtual machine specifications:
- Choose an availability zone. For Open Telekom Cloud reference see [here](https://docs.otc.t-systems.com/en-us/endpoint/index.html).
- Change the OS image (For T-Pot we need Debian)
- (Optional) Change the volume size
- Specify your key pair (:warning: Mandatory)
- (Optional) Change the instance type (flavor)
`s2.medium.8` corresponds to 1 vCPU and 8GB of RAM and is the minimum required flavor.
`s3.medium.8` corresponds to 1 vCPU and 8GB of RAM and is the minimum required flavor.
A full list of Open Telekom Cloud flavors can be found [here](https://docs.otc.t-systems.com/en-us/usermanual/ecs/en-us_topic_0177512565.html).
```
@ -149,7 +170,7 @@ availability_zone: eu-de-03
image: Standard_Debian_10_latest
volume_size: 128
key_name: your-KeyPair
flavor: s2.medium.8
flavor: s3.medium.8
```
<a name="user-password"></a>
@ -200,7 +221,7 @@ Enable this by uncommenting the role in the [deploy_tpot.yaml](openstack/deploy_
# - custom_hpfeeds
```
You can specify custom HPFEEDS in [`openstack/roles/custom_hpfeeds/templates/hpfeeds.cfg`](openstack/roles/custom_hpfeeds/templates/hpfeeds.cfg).
You can specify custom HPFEEDS in [`openstack/roles/custom_hpfeeds/files/hpfeeds.cfg`](openstack/roles/custom_hpfeeds/files/hpfeeds.cfg).
That file contains the defaults (turned off) and you can adapt it for your needs, e.g. for SISSDEN:
```
myENABLE=true
@ -216,6 +237,7 @@ myFORMAT=json
<a name="deploy"></a>
# Deploying a T-Pot :honey_pot::honeybee:
Now, after configuring everything, we can finally start deploying T-Pots!
Go to the [`openstack`](openstack) folder and run the Ansible Playbook with:
`ansible-playbook deploy_tpot.yaml`
(Yes, it is as easy as that :smile:)
@ -223,15 +245,13 @@ Go to the [`openstack`](openstack) folder and run the Ansible Playbook with:
If you are running on a machine which asks for a sudo password, you can use:
`ansible-playbook --ask-become-pass deploy_tpot.yaml`
The Playbook will first install required packages on the Ansible Master and then deploy a new server instance.
After that, T-Pot gets installed and configured on the newly created host, optionally custom configs are applied and finally it reboots.
The Playbook will first install required packages on the Ansible Master and then deploy one (or more) new server instances.
After that, T-Pot gets installed and configured on them, optionally custom configs are applied and finally it reboots.
Once this is done, you can proceed with connecting/logging in to the T-Pot according to the [documentation](https://github.com/telekom-security/tpotce#ssh-and-web-access).
<a name="documentation"></a>
# Further documentation
- [Ansible Documentation](https://docs.ansible.com/ansible/latest/)
- [Cloud modules — Ansible Documentation](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html)
- [os_server Create/Delete Compute Instances from OpenStack — Ansible Documentation](https://docs.ansible.com/ansible/latest/modules/os_server_module.html)
- [openstack.cloud.server Create/Delete Compute Instances from OpenStack](https://docs.ansible.com/ansible/latest/collections/openstack/cloud/server_module.html)
- [Open Telekom Cloud Help Center](https://docs.otc.t-systems.com/)
- [Open Telekom Cloud API Overview](https://docs.otc.t-systems.com/en-us/api/wp/en-us_topic_0052070394.html)

View file

@ -4,13 +4,22 @@
roles:
- check
- name: Deploy instance
- name: Deploy instances
hosts: localhost
roles:
- deploy
vars_files: my_os_cloud.yaml
tasks:
- name: Create security group and network
ansible.builtin.include_role:
name: create_net
- name: Create one or more instances
ansible.builtin.include_role:
name: create_vm
loop: "{{ range(0, 1) }}"
loop_control:
extended: yes
- name: Install T-Pot on new instance
hosts: TPOT
- name: Install T-Pot
hosts: tpot
remote_user: linux
become: yes
gather_facts: no

View file

@ -0,0 +1,2 @@
# Enter the name of your cloud to use from clouds.yaml
cloud: open-telekom-cloud

View file

@ -0,0 +1,2 @@
collections:
- name: openstack.cloud

View file

@ -1,20 +1,19 @@
- name: Install dependencies
package:
ansible.builtin.package:
name:
- gcc
- pwgen
- python3-dev
- python3-setuptools
- python3-pip
state: present
- name: Install openstacksdk
pip:
ansible.builtin.pip:
name: openstacksdk
executable: pip3
- name: Check if agent forwarding is enabled
fail:
ansible.builtin.fail:
msg: Please enable agent forwarding to allow Ansible to connect to the remote host!
ignore_errors: yes
when: lookup('env','SSH_AUTH_SOCK') == ""

View file

@ -0,0 +1,33 @@
- name: Create security group
openstack.cloud.security_group:
cloud: "{{ cloud }}"
name: sg-tpot-any
description: tpot any-any
- name: Add rules to security group
openstack.cloud.security_group_rule:
cloud: "{{ cloud }}"
security_group: sg-tpot-any
remote_ip_prefix: 0.0.0.0/0
- name: Create network
openstack.cloud.network:
cloud: "{{ cloud }}"
name: network-tpot
- name: Create subnet
openstack.cloud.subnet:
cloud: "{{ cloud }}"
network_name: network-tpot
name: subnet-tpot
cidr: 192.168.0.0/24
dns_nameservers:
- 1.1.1.1
- 8.8.8.8
- name: Create router
openstack.cloud.router:
cloud: "{{ cloud }}"
name: router-tpot
interfaces:
- subnet-tpot

View file

@ -0,0 +1,24 @@
- name: Generate T-Pot name
ansible.builtin.set_fact:
tpot_name: "t-pot-ansible-{{ lookup('password', '/dev/null chars=ascii_lowercase,digits length=6') }}"
- name: Create instance {{ ansible_loop.index }} of {{ ansible_loop.length }}
openstack.cloud.server:
cloud: "{{ cloud }}"
name: "{{ tpot_name }}"
availability_zone: "{{ availability_zone }}"
image: "{{ image }}"
boot_from_volume: yes
volume_size: "{{ volume_size }}"
key_name: "{{ key_name }}"
timeout: 200
flavor: "{{ flavor }}"
security_groups: sg-tpot-any
network: network-tpot
register: tpot
- name: Add instance to inventory
ansible.builtin.add_host:
hostname: "{{ tpot_name }}"
ansible_host: "{{ tpot.server.public_v4 }}"
groups: tpot

View file

@ -2,4 +2,4 @@ availability_zone: eu-de-03
image: Standard_Debian_10_latest
volume_size: 128
key_name: your-KeyPair
flavor: s2.medium.8
flavor: s3.medium.8

View file

@ -1,5 +1,5 @@
- name: Copy ews configuration file
template:
ansible.builtin.template:
src: ews.cfg
dest: /data/ews/conf
owner: root
@ -7,7 +7,7 @@
mode: 0644
- name: Patching tpot.yml with custom ews configuration file
lineinfile:
ansible.builtin.lineinfile:
path: /opt/tpot/etc/tpot.yml
insertafter: "/opt/ewsposter/ews.ip"
line: " - /data/ews/conf/ews.cfg:/opt/ewsposter/ews.cfg"

View file

@ -1,5 +1,5 @@
- name: Copy hpfeeds configuration file
copy:
ansible.builtin.copy:
src: hpfeeds.cfg
dest: /data/ews/conf
owner: tpot
@ -8,5 +8,5 @@
register: config
- name: Applying hpfeeds settings
command: /opt/tpot/bin/hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg
ansible.builtin.command: /opt/tpot/bin/hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg
when: config.changed == true

View file

@ -1,58 +0,0 @@
- name: Create T-Pot name
shell: echo t-pot-ansible-$(pwgen -ns 6 -1)
register: tpot_name
- name: Create security group
os_security_group:
cloud: open-telekom-cloud
name: sg-tpot-any
description: tpot any-any
- name: Add rules to security group
os_security_group_rule:
cloud: open-telekom-cloud
security_group: sg-tpot-any
remote_ip_prefix: 0.0.0.0/0
- name: Create network
os_network:
cloud: open-telekom-cloud
name: network-tpot
- name: Create subnet
os_subnet:
cloud: open-telekom-cloud
network_name: network-tpot
name: subnet-tpot
cidr: 192.168.0.0/24
dns_nameservers:
- 1.1.1.1
- 8.8.8.8
- name: Create router
os_router:
cloud: open-telekom-cloud
name: router-tpot
interfaces:
- subnet-tpot
- name: Launch an instance
os_server:
cloud: open-telekom-cloud
name: "{{ tpot_name.stdout }}"
availability_zone: "{{ availability_zone }}"
image: "{{ image }}"
boot_from_volume: yes
volume_size: "{{ volume_size }}"
key_name: "{{ key_name }}"
timeout: 200
flavor: "{{ flavor }}"
security_groups: sg-tpot-any
network: network-tpot
register: tpot
- name: Add instance to inventory
add_host:
hostname: "{{ tpot_name.stdout }}"
ansible_host: "{{ tpot.server.public_v4 }}"
groups: TPOT

View file

@ -1,29 +1,29 @@
- name: Waiting for SSH connection
wait_for_connection:
ansible.builtin.wait_for_connection:
- name: Gathering facts
setup:
ansible.builtin.setup:
- name: Cloning T-Pot install directory
git:
ansible.builtin.git:
repo: "https://github.com/telekom-security/tpotce.git"
dest: /root/tpot
- name: Prepare to set user password
set_fact:
ansible.builtin.set_fact:
user_name: "{{ ansible_user }}"
user_salt: "s0mew1ck3dTpoT"
no_log: true
- name: Changing password for user {{ user_name }}
user:
ansible.builtin.user:
name: "{{ ansible_user }}"
password: "{{ user_password | password_hash('sha512', user_salt) }}"
state: present
shell: /bin/bash
- name: Copy T-Pot configuration file
template:
ansible.builtin.template:
src: ../../../../../../iso/installer/tpot.conf.dist
dest: /root/tpot.conf
owner: root
@ -31,15 +31,15 @@
mode: 0644
- name: Install T-Pot on instance - be patient, this might take 15 to 30 minutes depending on the connection speed.
command: /root/tpot/iso/installer/install.sh --type=auto --conf=/root/tpot.conf
ansible.builtin.command: /root/tpot/iso/installer/install.sh --type=auto --conf=/root/tpot.conf
- name: Delete T-Pot configuration file
file:
ansible.builtin.file:
path: /root/tpot.conf
state: absent
- name: Change unattended-upgrades to take default action
blockinfile:
ansible.builtin.blockinfile:
dest: /etc/apt/apt.conf.d/50unattended-upgrades
block: |
Dpkg::Options {

View file

@ -1,10 +1,10 @@
- name: Finally rebooting T-Pot
command: shutdown -r now
ansible.builtin.command: shutdown -r now
async: 1
poll: 0
- name: Next login options
debug:
ansible.builtin.debug:
msg:
- "***** SSH Access:"
- "***** ssh {{ ansible_user }}@{{ ansible_host }} -p 64295"

View file

@ -1,2 +0,0 @@
**/.terraform
**/terraform.*

View file

@ -1,7 +1,7 @@
# T-Pot Terraform
This [Terraform](https://www.terraform.io/) configuration can be used to launch a virtual machine, bootstrap any dependencies and install T-Pot in a single step.
Configuration for Amazon Web Services (AWS) and Open Telekom Cloud (OTC) is currently included.
This can easily be extended to support other [Terraform providers](https://www.terraform.io/docs/providers/index.html).
This can easily be extended to support other [Terraform providers](https://registry.terraform.io/browse/providers?category=public-cloud%2Ccloud-automation%2Cinfrastructure).
[Cloud-init](https://cloudinit.readthedocs.io/en/latest/) is used to bootstrap the instance and install T-Pot on startup.