diff --git a/README.md b/README.md index e6fcbe6b..d5135241 100644 --- a/README.md +++ b/README.md @@ -290,9 +290,9 @@ If you would like to contribute, you can add other cloud deployments like Chef o You can find an [Ansible](https://www.ansible.com/) based T-Pot deployment in the [`cloud/ansible`](cloud/ansible) folder. The Playbook in the [`cloud/ansible/openstack`](cloud/ansible/openstack) folder is reusable for all **OpenStack** clouds out of the box. -It first creates all resources (security group, network, subnet, router), deploys a new server and then installs and configures T-Pot. +It first creates all resources (security group, network, subnet, router), deploys one (or more) new servers and then installs and configures T-Pot on them. -You can have a look at the Playbook and easily adapt the deploy role for other [cloud providers](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html). +You can have a look at the Playbook and easily adapt the deploy role for other [cloud providers](https://docs.ansible.com/ansible/latest/scenario_guides/cloud_guides.html). Check out [Ansible Galaxy](https://galaxy.ansible.com/search?keywords=&order_by=-relevance&page=1&deprecated=false&type=collection&tags=cloud) for more cloud collections. *Please note*: Cloud providers usually offer adjusted Debian OS images, which might not be compatible with T-Pot. There is no cloud provider support provided of any kind. @@ -304,7 +304,7 @@ You can find [Terraform](https://www.terraform.io/) configuration in the [`cloud This can be used to launch a virtual machine, bootstrap any dependencies and install T-Pot in a single step. Configuration for **Amazon Web Services** (AWS) and **Open Telekom Cloud** (OTC) is currently included. -This can easily be extended to support other [Terraform providers](https://www.terraform.io/docs/providers/index.html). +This can easily be extended to support other [Terraform providers](https://registry.terraform.io/browse/providers?category=public-cloud%2Ccloud-automation%2Cinfrastructure). *Please note*: Cloud providers usually offer adjusted Debian OS images, which might not be compatible with T-Pot. There is no cloud provider support provided of any kind. diff --git a/cloud/.gitignore b/cloud/.gitignore new file mode 100644 index 00000000..06ddc876 --- /dev/null +++ b/cloud/.gitignore @@ -0,0 +1,10 @@ +# Ansible +*.retry + +# Terraform +**/.terraform +**/terraform.* + +# OpenStack clouds +clouds.yaml +secure.yaml diff --git a/cloud/ansible/.gitignore b/cloud/ansible/.gitignore deleted file mode 100644 index 41f83492..00000000 --- a/cloud/ansible/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -# Ansible -*.retry diff --git a/cloud/ansible/README.md b/cloud/ansible/README.md index 72fb7026..5be6a912 100644 --- a/cloud/ansible/README.md +++ b/cloud/ansible/README.md @@ -2,15 +2,16 @@ Here you can find a ready-to-use solution for your automated T-Pot deployment using [Ansible](https://www.ansible.com/). It consists of an Ansible Playbook with multiple roles, which is reusable for all [OpenStack](https://www.openstack.org/) based clouds (e.g. Open Telekom Cloud, Orange Cloud, Telefonica Open Cloud, OVH) out of the box. -Apart from that you can easily adapt the deploy role to use other [cloud providers](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html) (e.g. AWS, Azure, Digital Ocean, Google). +Apart from that you can easily adapt the deploy role to use other [cloud providers](https://docs.ansible.com/ansible/latest/scenario_guides/cloud_guides.html). Check out [Ansible Galaxy](https://galaxy.ansible.com/search?keywords=&order_by=-relevance&page=1&deprecated=false&type=collection&tags=cloud) for more cloud collections. -The Playbook first creates all resources (security group, network, subnet, router), deploys a new server and then installs and configures T-Pot. +The Playbook first creates all resources (security group, network, subnet, router), deploys one (or more) new servers and then installs and configures T-Pot on them. This example showcases the deployment on our own OpenStack based Public Cloud Offering [Open Telekom Cloud](https://open-telekom-cloud.com/en). # Table of contents - [Preparation of Ansible Master](#ansible-master) - [Ansible Installation](#ansible) + - [OpenStack Collection Installation](#collection) - [Agent Forwarding](#agent-forwarding) - [Preparations in Open Telekom Cloud Console](#preparation) - [Create new project](#project) @@ -18,8 +19,9 @@ This example showcases the deployment on our own OpenStack based Public Cloud Of - [Import Key Pair](#key-pair) - [Clone Git Repository](#clone-git) - [Settings and recommended values](#settings) - - [Clouds.yaml](#clouds-yaml) + - [clouds.yaml](#clouds-yaml) - [Ansible remote user](#remote-user) + - [Number of instances to deploy](#number) - [Instance settings](#instance-settings) - [User password](#user-password) - [Configure `tpot.conf.dist`](#tpot-conf) @@ -56,6 +58,11 @@ In short (if you already have Python3/pip3 installed): pip3 install ansible ``` + +## OpenStack Collection Installation +For interacting with OpenStack resources in Ansible, you need to install the collection from Ansible Galaxy: +`ansible-galaxy collection install openstack.cloud` + ## Agent Forwarding If you run the Ansible Playbook remotely on your Ansible Master Server, Agent Forwarding must be enabled in order to let Ansible connect to newly created machines. @@ -112,7 +119,7 @@ All Ansible related files are located in the [`cloud/ansible/openstack`](opensta You can configure all aspects of your Elastic Cloud Server and T-Pot before using the Playbook: -## Clouds.yaml +## clouds.yaml Located at [`openstack/clouds.yaml`](openstack/clouds.yaml). Enter your Open Telekom Cloud API user credentials here (username, password, project name, user domain name): ``` @@ -126,22 +133,36 @@ clouds: user_domain_name: OTC-EU-DE-000000000010000XXXXX ``` You can also perform different authentication methods like sourcing OpenStack OS_* environment variables or providing an inline dictionary. -For more information have a look in the [os_server](https://docs.ansible.com/ansible/latest/modules/os_server_module.html) Ansible module documentation. +For more information have a look in the [openstack.cloud.server](https://docs.ansible.com/ansible/latest/collections/openstack/cloud/server_module.html) Ansible module documentation. + +If you already have your own `clouds.yaml` file or have multiple clouds in there, you can specify which one to use in the `openstack/my_os_cloud.yaml` file: +``` +# Enter the name of your cloud to use from clouds.yaml +cloud: open-telekom-cloud +``` ## Ansible remote user You may have to adjust the `remote_user` in the Ansible Playbook under [`openstack/deploy_tpot.yaml`](openstack/deploy_tpot.yaml) depending on your Debian base image (e.g. on Open Telekom Cloud the default Debian user is `linux`). + +## Number of instances to deploy +You can adjust the number of VMs/T-Pots that you want to create in [`openstack/deploy_tpot.yaml`](openstack/deploy_tpot.yaml): +``` +loop: "{{ range(0, 1) }}" +``` +One instance is set as the default, increase to your liking. + ## Instance settings -Located at [`openstack/roles/deploy/vars/main.yaml`](openstack/roles/deploy/vars/main.yaml). +Located at [`openstack/roles/create_vm/vars/main.yaml`](openstack/roles/create_vm/vars/main.yaml). Here you can customize your virtual machine specifications: - Choose an availability zone. For Open Telekom Cloud reference see [here](https://docs.otc.t-systems.com/en-us/endpoint/index.html). - Change the OS image (For T-Pot we need Debian) - (Optional) Change the volume size - Specify your key pair (:warning: Mandatory) - (Optional) Change the instance type (flavor) - `s2.medium.8` corresponds to 1 vCPU and 8GB of RAM and is the minimum required flavor. + `s3.medium.8` corresponds to 1 vCPU and 8GB of RAM and is the minimum required flavor. A full list of Open Telekom Cloud flavors can be found [here](https://docs.otc.t-systems.com/en-us/usermanual/ecs/en-us_topic_0177512565.html). ``` @@ -149,7 +170,7 @@ availability_zone: eu-de-03 image: Standard_Debian_10_latest volume_size: 128 key_name: your-KeyPair -flavor: s2.medium.8 +flavor: s3.medium.8 ``` @@ -200,7 +221,7 @@ Enable this by uncommenting the role in the [deploy_tpot.yaml](openstack/deploy_ # - custom_hpfeeds ``` -You can specify custom HPFEEDS in [`openstack/roles/custom_hpfeeds/templates/hpfeeds.cfg`](openstack/roles/custom_hpfeeds/templates/hpfeeds.cfg). +You can specify custom HPFEEDS in [`openstack/roles/custom_hpfeeds/files/hpfeeds.cfg`](openstack/roles/custom_hpfeeds/files/hpfeeds.cfg). That file contains the defaults (turned off) and you can adapt it for your needs, e.g. for SISSDEN: ``` myENABLE=true @@ -216,6 +237,7 @@ myFORMAT=json # Deploying a T-Pot :honey_pot::honeybee: Now, after configuring everything, we can finally start deploying T-Pots! + Go to the [`openstack`](openstack) folder and run the Ansible Playbook with: `ansible-playbook deploy_tpot.yaml` (Yes, it is as easy as that :smile:) @@ -223,15 +245,13 @@ Go to the [`openstack`](openstack) folder and run the Ansible Playbook with: If you are running on a machine which asks for a sudo password, you can use: `ansible-playbook --ask-become-pass deploy_tpot.yaml` -The Playbook will first install required packages on the Ansible Master and then deploy a new server instance. -After that, T-Pot gets installed and configured on the newly created host, optionally custom configs are applied and finally it reboots. +The Playbook will first install required packages on the Ansible Master and then deploy one (or more) new server instances. +After that, T-Pot gets installed and configured on them, optionally custom configs are applied and finally it reboots. Once this is done, you can proceed with connecting/logging in to the T-Pot according to the [documentation](https://github.com/telekom-security/tpotce#ssh-and-web-access). # Further documentation - [Ansible Documentation](https://docs.ansible.com/ansible/latest/) -- [Cloud modules — Ansible Documentation](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html) -- [os_server – Create/Delete Compute Instances from OpenStack — Ansible Documentation](https://docs.ansible.com/ansible/latest/modules/os_server_module.html) +- [openstack.cloud.server – Create/Delete Compute Instances from OpenStack](https://docs.ansible.com/ansible/latest/collections/openstack/cloud/server_module.html) - [Open Telekom Cloud Help Center](https://docs.otc.t-systems.com/) -- [Open Telekom Cloud API Overview](https://docs.otc.t-systems.com/en-us/api/wp/en-us_topic_0052070394.html) diff --git a/cloud/ansible/openstack/deploy_tpot.yaml b/cloud/ansible/openstack/deploy_tpot.yaml index c06ef4c5..5e3ee05a 100644 --- a/cloud/ansible/openstack/deploy_tpot.yaml +++ b/cloud/ansible/openstack/deploy_tpot.yaml @@ -4,13 +4,22 @@ roles: - check -- name: Deploy instance +- name: Deploy instances hosts: localhost - roles: - - deploy + vars_files: my_os_cloud.yaml + tasks: + - name: Create security group and network + ansible.builtin.include_role: + name: create_net + - name: Create one or more instances + ansible.builtin.include_role: + name: create_vm + loop: "{{ range(0, 1) }}" + loop_control: + extended: yes -- name: Install T-Pot on new instance - hosts: TPOT +- name: Install T-Pot + hosts: tpot remote_user: linux become: yes gather_facts: no diff --git a/cloud/ansible/openstack/my_os_cloud.yaml b/cloud/ansible/openstack/my_os_cloud.yaml new file mode 100644 index 00000000..d3832f85 --- /dev/null +++ b/cloud/ansible/openstack/my_os_cloud.yaml @@ -0,0 +1,2 @@ +# Enter the name of your cloud to use from clouds.yaml +cloud: open-telekom-cloud diff --git a/cloud/ansible/openstack/requirements.yaml b/cloud/ansible/openstack/requirements.yaml new file mode 100644 index 00000000..986ae0e5 --- /dev/null +++ b/cloud/ansible/openstack/requirements.yaml @@ -0,0 +1,2 @@ +collections: +- name: openstack.cloud diff --git a/cloud/ansible/openstack/roles/check/tasks/main.yaml b/cloud/ansible/openstack/roles/check/tasks/main.yaml index d9483ef4..3c2ac286 100644 --- a/cloud/ansible/openstack/roles/check/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/check/tasks/main.yaml @@ -1,20 +1,19 @@ - name: Install dependencies - package: + ansible.builtin.package: name: - gcc - - pwgen - python3-dev - python3-setuptools - python3-pip state: present - name: Install openstacksdk - pip: + ansible.builtin.pip: name: openstacksdk executable: pip3 - name: Check if agent forwarding is enabled - fail: + ansible.builtin.fail: msg: Please enable agent forwarding to allow Ansible to connect to the remote host! ignore_errors: yes when: lookup('env','SSH_AUTH_SOCK') == "" diff --git a/cloud/ansible/openstack/roles/create_net/tasks/main.yaml b/cloud/ansible/openstack/roles/create_net/tasks/main.yaml new file mode 100644 index 00000000..c372d8bb --- /dev/null +++ b/cloud/ansible/openstack/roles/create_net/tasks/main.yaml @@ -0,0 +1,33 @@ +- name: Create security group + openstack.cloud.security_group: + cloud: "{{ cloud }}" + name: sg-tpot-any + description: tpot any-any + +- name: Add rules to security group + openstack.cloud.security_group_rule: + cloud: "{{ cloud }}" + security_group: sg-tpot-any + remote_ip_prefix: 0.0.0.0/0 + +- name: Create network + openstack.cloud.network: + cloud: "{{ cloud }}" + name: network-tpot + +- name: Create subnet + openstack.cloud.subnet: + cloud: "{{ cloud }}" + network_name: network-tpot + name: subnet-tpot + cidr: 192.168.0.0/24 + dns_nameservers: + - 1.1.1.1 + - 8.8.8.8 + +- name: Create router + openstack.cloud.router: + cloud: "{{ cloud }}" + name: router-tpot + interfaces: + - subnet-tpot diff --git a/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml b/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml new file mode 100644 index 00000000..71ad71e7 --- /dev/null +++ b/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml @@ -0,0 +1,24 @@ +- name: Generate T-Pot name + ansible.builtin.set_fact: + tpot_name: "t-pot-ansible-{{ lookup('password', '/dev/null chars=ascii_lowercase,digits length=6') }}" + +- name: Create instance {{ ansible_loop.index }} of {{ ansible_loop.length }} + openstack.cloud.server: + cloud: "{{ cloud }}" + name: "{{ tpot_name }}" + availability_zone: "{{ availability_zone }}" + image: "{{ image }}" + boot_from_volume: yes + volume_size: "{{ volume_size }}" + key_name: "{{ key_name }}" + timeout: 200 + flavor: "{{ flavor }}" + security_groups: sg-tpot-any + network: network-tpot + register: tpot + +- name: Add instance to inventory + ansible.builtin.add_host: + hostname: "{{ tpot_name }}" + ansible_host: "{{ tpot.server.public_v4 }}" + groups: tpot diff --git a/cloud/ansible/openstack/roles/deploy/vars/main.yaml b/cloud/ansible/openstack/roles/create_vm/vars/main.yaml similarity index 83% rename from cloud/ansible/openstack/roles/deploy/vars/main.yaml rename to cloud/ansible/openstack/roles/create_vm/vars/main.yaml index d2b0664a..cd56d25f 100644 --- a/cloud/ansible/openstack/roles/deploy/vars/main.yaml +++ b/cloud/ansible/openstack/roles/create_vm/vars/main.yaml @@ -2,4 +2,4 @@ availability_zone: eu-de-03 image: Standard_Debian_10_latest volume_size: 128 key_name: your-KeyPair -flavor: s2.medium.8 +flavor: s3.medium.8 diff --git a/cloud/ansible/openstack/roles/custom_ews/tasks/main.yaml b/cloud/ansible/openstack/roles/custom_ews/tasks/main.yaml index 1856a3df..fec93410 100644 --- a/cloud/ansible/openstack/roles/custom_ews/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/custom_ews/tasks/main.yaml @@ -1,5 +1,5 @@ - name: Copy ews configuration file - template: + ansible.builtin.template: src: ews.cfg dest: /data/ews/conf owner: root @@ -7,7 +7,7 @@ mode: 0644 - name: Patching tpot.yml with custom ews configuration file - lineinfile: + ansible.builtin.lineinfile: path: /opt/tpot/etc/tpot.yml insertafter: "/opt/ewsposter/ews.ip" line: " - /data/ews/conf/ews.cfg:/opt/ewsposter/ews.cfg" diff --git a/cloud/ansible/openstack/roles/custom_hpfeeds/tasks/main.yaml b/cloud/ansible/openstack/roles/custom_hpfeeds/tasks/main.yaml index 50ea7311..fa479137 100644 --- a/cloud/ansible/openstack/roles/custom_hpfeeds/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/custom_hpfeeds/tasks/main.yaml @@ -1,5 +1,5 @@ - name: Copy hpfeeds configuration file - copy: + ansible.builtin.copy: src: hpfeeds.cfg dest: /data/ews/conf owner: tpot @@ -8,5 +8,5 @@ register: config - name: Applying hpfeeds settings - command: /opt/tpot/bin/hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg + ansible.builtin.command: /opt/tpot/bin/hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg when: config.changed == true diff --git a/cloud/ansible/openstack/roles/deploy/tasks/main.yaml b/cloud/ansible/openstack/roles/deploy/tasks/main.yaml deleted file mode 100644 index bd68b1bf..00000000 --- a/cloud/ansible/openstack/roles/deploy/tasks/main.yaml +++ /dev/null @@ -1,58 +0,0 @@ -- name: Create T-Pot name - shell: echo t-pot-ansible-$(pwgen -ns 6 -1) - register: tpot_name - -- name: Create security group - os_security_group: - cloud: open-telekom-cloud - name: sg-tpot-any - description: tpot any-any - -- name: Add rules to security group - os_security_group_rule: - cloud: open-telekom-cloud - security_group: sg-tpot-any - remote_ip_prefix: 0.0.0.0/0 - -- name: Create network - os_network: - cloud: open-telekom-cloud - name: network-tpot - -- name: Create subnet - os_subnet: - cloud: open-telekom-cloud - network_name: network-tpot - name: subnet-tpot - cidr: 192.168.0.0/24 - dns_nameservers: - - 1.1.1.1 - - 8.8.8.8 - -- name: Create router - os_router: - cloud: open-telekom-cloud - name: router-tpot - interfaces: - - subnet-tpot - -- name: Launch an instance - os_server: - cloud: open-telekom-cloud - name: "{{ tpot_name.stdout }}" - availability_zone: "{{ availability_zone }}" - image: "{{ image }}" - boot_from_volume: yes - volume_size: "{{ volume_size }}" - key_name: "{{ key_name }}" - timeout: 200 - flavor: "{{ flavor }}" - security_groups: sg-tpot-any - network: network-tpot - register: tpot - -- name: Add instance to inventory - add_host: - hostname: "{{ tpot_name.stdout }}" - ansible_host: "{{ tpot.server.public_v4 }}" - groups: TPOT diff --git a/cloud/ansible/openstack/roles/install/tasks/main.yaml b/cloud/ansible/openstack/roles/install/tasks/main.yaml index 173c4f08..3338294d 100644 --- a/cloud/ansible/openstack/roles/install/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/install/tasks/main.yaml @@ -1,29 +1,29 @@ - name: Waiting for SSH connection - wait_for_connection: + ansible.builtin.wait_for_connection: - name: Gathering facts - setup: + ansible.builtin.setup: - name: Cloning T-Pot install directory - git: + ansible.builtin.git: repo: "https://github.com/telekom-security/tpotce.git" dest: /root/tpot - name: Prepare to set user password - set_fact: + ansible.builtin.set_fact: user_name: "{{ ansible_user }}" user_salt: "s0mew1ck3dTpoT" no_log: true - name: Changing password for user {{ user_name }} - user: + ansible.builtin.user: name: "{{ ansible_user }}" password: "{{ user_password | password_hash('sha512', user_salt) }}" state: present shell: /bin/bash - name: Copy T-Pot configuration file - template: + ansible.builtin.template: src: ../../../../../../iso/installer/tpot.conf.dist dest: /root/tpot.conf owner: root @@ -31,15 +31,15 @@ mode: 0644 - name: Install T-Pot on instance - be patient, this might take 15 to 30 minutes depending on the connection speed. - command: /root/tpot/iso/installer/install.sh --type=auto --conf=/root/tpot.conf + ansible.builtin.command: /root/tpot/iso/installer/install.sh --type=auto --conf=/root/tpot.conf - name: Delete T-Pot configuration file - file: + ansible.builtin.file: path: /root/tpot.conf state: absent - name: Change unattended-upgrades to take default action - blockinfile: + ansible.builtin.blockinfile: dest: /etc/apt/apt.conf.d/50unattended-upgrades block: | Dpkg::Options { diff --git a/cloud/ansible/openstack/roles/reboot/tasks/main.yaml b/cloud/ansible/openstack/roles/reboot/tasks/main.yaml index 946d38d1..1d5ce5d5 100644 --- a/cloud/ansible/openstack/roles/reboot/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/reboot/tasks/main.yaml @@ -1,10 +1,10 @@ - name: Finally rebooting T-Pot - command: shutdown -r now + ansible.builtin.command: shutdown -r now async: 1 poll: 0 - name: Next login options - debug: + ansible.builtin.debug: msg: - "***** SSH Access:" - "***** ssh {{ ansible_user }}@{{ ansible_host }} -p 64295" diff --git a/cloud/terraform/.gitignore b/cloud/terraform/.gitignore deleted file mode 100644 index bcb28067..00000000 --- a/cloud/terraform/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -**/.terraform -**/terraform.* diff --git a/cloud/terraform/README.md b/cloud/terraform/README.md index 895a2af5..b8e19900 100644 --- a/cloud/terraform/README.md +++ b/cloud/terraform/README.md @@ -1,7 +1,7 @@ # T-Pot Terraform This [Terraform](https://www.terraform.io/) configuration can be used to launch a virtual machine, bootstrap any dependencies and install T-Pot in a single step. Configuration for Amazon Web Services (AWS) and Open Telekom Cloud (OTC) is currently included. -This can easily be extended to support other [Terraform providers](https://www.terraform.io/docs/providers/index.html). +This can easily be extended to support other [Terraform providers](https://registry.terraform.io/browse/providers?category=public-cloud%2Ccloud-automation%2Cinfrastructure). [Cloud-init](https://cloudinit.readthedocs.io/en/latest/) is used to bootstrap the instance and install T-Pot on startup.