From 51b15b65108945ccd87e7692442f858e930b1ab8 Mon Sep 17 00:00:00 2001 From: Sebastian Haderecker Date: Sat, 13 Feb 2021 15:04:50 +0100 Subject: [PATCH 01/15] Update docu links --- README.md | 4 ++-- cloud/ansible/README.md | 6 ++---- cloud/terraform/README.md | 2 +- 3 files changed, 5 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index e6fcbe6b..26ae8dfa 100644 --- a/README.md +++ b/README.md @@ -292,7 +292,7 @@ The Playbook in the [`cloud/ansible/openstack`](cloud/ansible/openstack) folder It first creates all resources (security group, network, subnet, router), deploys a new server and then installs and configures T-Pot. -You can have a look at the Playbook and easily adapt the deploy role for other [cloud providers](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html). +You can have a look at the Playbook and easily adapt the deploy role for other [cloud providers](https://docs.ansible.com/ansible/latest/scenario_guides/cloud_guides.html). Check out [Ansible Galaxy](https://galaxy.ansible.com/search?keywords=&order_by=-relevance&page=1&deprecated=false&type=collection&tags=cloud) for more cloud collections. *Please note*: Cloud providers usually offer adjusted Debian OS images, which might not be compatible with T-Pot. There is no cloud provider support provided of any kind. @@ -304,7 +304,7 @@ You can find [Terraform](https://www.terraform.io/) configuration in the [`cloud This can be used to launch a virtual machine, bootstrap any dependencies and install T-Pot in a single step. Configuration for **Amazon Web Services** (AWS) and **Open Telekom Cloud** (OTC) is currently included. -This can easily be extended to support other [Terraform providers](https://www.terraform.io/docs/providers/index.html). +This can easily be extended to support other [Terraform providers](https://registry.terraform.io/browse/providers?category=public-cloud%2Ccloud-automation%2Cinfrastructure). *Please note*: Cloud providers usually offer adjusted Debian OS images, which might not be compatible with T-Pot. There is no cloud provider support provided of any kind. diff --git a/cloud/ansible/README.md b/cloud/ansible/README.md index 72fb7026..05d94819 100644 --- a/cloud/ansible/README.md +++ b/cloud/ansible/README.md @@ -2,7 +2,7 @@ Here you can find a ready-to-use solution for your automated T-Pot deployment using [Ansible](https://www.ansible.com/). It consists of an Ansible Playbook with multiple roles, which is reusable for all [OpenStack](https://www.openstack.org/) based clouds (e.g. Open Telekom Cloud, Orange Cloud, Telefonica Open Cloud, OVH) out of the box. -Apart from that you can easily adapt the deploy role to use other [cloud providers](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html) (e.g. AWS, Azure, Digital Ocean, Google). +Apart from that you can easily adapt the deploy role to use other [cloud providers](https://docs.ansible.com/ansible/latest/scenario_guides/cloud_guides.html). Check out [Ansible Galaxy](https://galaxy.ansible.com/search?keywords=&order_by=-relevance&page=1&deprecated=false&type=collection&tags=cloud) for more cloud collections. The Playbook first creates all resources (security group, network, subnet, router), deploys a new server and then installs and configures T-Pot. @@ -231,7 +231,5 @@ Once this is done, you can proceed with connecting/logging in to the T-Pot accor # Further documentation - [Ansible Documentation](https://docs.ansible.com/ansible/latest/) -- [Cloud modules — Ansible Documentation](https://docs.ansible.com/ansible/latest/modules/list_of_cloud_modules.html) -- [os_server – Create/Delete Compute Instances from OpenStack — Ansible Documentation](https://docs.ansible.com/ansible/latest/modules/os_server_module.html) +- [openstack.cloud.server – Create/Delete Compute Instances from OpenStack](https://docs.ansible.com/ansible/latest/collections/openstack/cloud/server_module.html) - [Open Telekom Cloud Help Center](https://docs.otc.t-systems.com/) -- [Open Telekom Cloud API Overview](https://docs.otc.t-systems.com/en-us/api/wp/en-us_topic_0052070394.html) diff --git a/cloud/terraform/README.md b/cloud/terraform/README.md index 895a2af5..b8e19900 100644 --- a/cloud/terraform/README.md +++ b/cloud/terraform/README.md @@ -1,7 +1,7 @@ # T-Pot Terraform This [Terraform](https://www.terraform.io/) configuration can be used to launch a virtual machine, bootstrap any dependencies and install T-Pot in a single step. Configuration for Amazon Web Services (AWS) and Open Telekom Cloud (OTC) is currently included. -This can easily be extended to support other [Terraform providers](https://www.terraform.io/docs/providers/index.html). +This can easily be extended to support other [Terraform providers](https://registry.terraform.io/browse/providers?category=public-cloud%2Ccloud-automation%2Cinfrastructure). [Cloud-init](https://cloudinit.readthedocs.io/en/latest/) is used to bootstrap the instance and install T-Pot on startup. From c3ebf8487b9cf9b4f4b65efa87ad99b64dcf2345 Mon Sep 17 00:00:00 2001 From: Sebastian Haderecker Date: Sat, 13 Feb 2021 15:27:36 +0100 Subject: [PATCH 02/15] Lowercase group names --- cloud/ansible/openstack/deploy_tpot.yaml | 2 +- cloud/ansible/openstack/roles/deploy/tasks/main.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/cloud/ansible/openstack/deploy_tpot.yaml b/cloud/ansible/openstack/deploy_tpot.yaml index c06ef4c5..7e58209b 100644 --- a/cloud/ansible/openstack/deploy_tpot.yaml +++ b/cloud/ansible/openstack/deploy_tpot.yaml @@ -10,7 +10,7 @@ - deploy - name: Install T-Pot on new instance - hosts: TPOT + hosts: tpot remote_user: linux become: yes gather_facts: no diff --git a/cloud/ansible/openstack/roles/deploy/tasks/main.yaml b/cloud/ansible/openstack/roles/deploy/tasks/main.yaml index bd68b1bf..36290db2 100644 --- a/cloud/ansible/openstack/roles/deploy/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/deploy/tasks/main.yaml @@ -55,4 +55,4 @@ add_host: hostname: "{{ tpot_name.stdout }}" ansible_host: "{{ tpot.server.public_v4 }}" - groups: TPOT + groups: tpot From 487c091ba7aef76574c8660b8e5285cea4538586 Mon Sep 17 00:00:00 2001 From: Sebastian Haderecker Date: Sat, 13 Feb 2021 15:36:39 +0100 Subject: [PATCH 03/15] Use ansible internal tools to generate random name --- cloud/ansible/openstack/roles/check/tasks/main.yaml | 1 - cloud/ansible/openstack/roles/deploy/tasks/main.yaml | 6 +++--- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/cloud/ansible/openstack/roles/check/tasks/main.yaml b/cloud/ansible/openstack/roles/check/tasks/main.yaml index d9483ef4..fd58fc6a 100644 --- a/cloud/ansible/openstack/roles/check/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/check/tasks/main.yaml @@ -2,7 +2,6 @@ package: name: - gcc - - pwgen - python3-dev - python3-setuptools - python3-pip diff --git a/cloud/ansible/openstack/roles/deploy/tasks/main.yaml b/cloud/ansible/openstack/roles/deploy/tasks/main.yaml index 36290db2..dda25d7b 100644 --- a/cloud/ansible/openstack/roles/deploy/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/deploy/tasks/main.yaml @@ -1,6 +1,6 @@ -- name: Create T-Pot name - shell: echo t-pot-ansible-$(pwgen -ns 6 -1) - register: tpot_name +- name: Generate T-Pot name + set_fact: + tpot_name: "t-pot-ansible-{{ lookup('password', '/dev/null chars=ascii_lowercase,digits length=6') }}" - name: Create security group os_security_group: From bb8d2f27c6989e3037afa98ad547bae590888216 Mon Sep 17 00:00:00 2001 From: Sebastian Haderecker Date: Sat, 13 Feb 2021 17:22:49 +0100 Subject: [PATCH 04/15] Split network and vm creation into own roles --- cloud/ansible/openstack/deploy_tpot.yaml | 3 ++- .../{deploy => create_net}/tasks/main.yaml | 25 ------------------- .../openstack/roles/create_vm/tasks/main.yaml | 24 ++++++++++++++++++ .../{deploy => create_vm}/vars/main.yaml | 2 +- 4 files changed, 27 insertions(+), 27 deletions(-) rename cloud/ansible/openstack/roles/{deploy => create_net}/tasks/main.yaml (50%) create mode 100644 cloud/ansible/openstack/roles/create_vm/tasks/main.yaml rename cloud/ansible/openstack/roles/{deploy => create_vm}/vars/main.yaml (83%) diff --git a/cloud/ansible/openstack/deploy_tpot.yaml b/cloud/ansible/openstack/deploy_tpot.yaml index 7e58209b..9bcc22a8 100644 --- a/cloud/ansible/openstack/deploy_tpot.yaml +++ b/cloud/ansible/openstack/deploy_tpot.yaml @@ -7,7 +7,8 @@ - name: Deploy instance hosts: localhost roles: - - deploy + - create_net + - create_vm - name: Install T-Pot on new instance hosts: tpot diff --git a/cloud/ansible/openstack/roles/deploy/tasks/main.yaml b/cloud/ansible/openstack/roles/create_net/tasks/main.yaml similarity index 50% rename from cloud/ansible/openstack/roles/deploy/tasks/main.yaml rename to cloud/ansible/openstack/roles/create_net/tasks/main.yaml index dda25d7b..970df2d0 100644 --- a/cloud/ansible/openstack/roles/deploy/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/create_net/tasks/main.yaml @@ -1,7 +1,3 @@ -- name: Generate T-Pot name - set_fact: - tpot_name: "t-pot-ansible-{{ lookup('password', '/dev/null chars=ascii_lowercase,digits length=6') }}" - - name: Create security group os_security_group: cloud: open-telekom-cloud @@ -35,24 +31,3 @@ name: router-tpot interfaces: - subnet-tpot - -- name: Launch an instance - os_server: - cloud: open-telekom-cloud - name: "{{ tpot_name.stdout }}" - availability_zone: "{{ availability_zone }}" - image: "{{ image }}" - boot_from_volume: yes - volume_size: "{{ volume_size }}" - key_name: "{{ key_name }}" - timeout: 200 - flavor: "{{ flavor }}" - security_groups: sg-tpot-any - network: network-tpot - register: tpot - -- name: Add instance to inventory - add_host: - hostname: "{{ tpot_name.stdout }}" - ansible_host: "{{ tpot.server.public_v4 }}" - groups: tpot diff --git a/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml b/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml new file mode 100644 index 00000000..c04be549 --- /dev/null +++ b/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml @@ -0,0 +1,24 @@ +- name: Generate T-Pot name + set_fact: + tpot_name: "t-pot-ansible-{{ lookup('password', '/dev/null chars=ascii_lowercase,digits length=6') }}" + +- name: Create instance + os_server: + cloud: open-telekom-cloud + name: "{{ tpot_name.stdout }}" + availability_zone: "{{ availability_zone }}" + image: "{{ image }}" + boot_from_volume: yes + volume_size: "{{ volume_size }}" + key_name: "{{ key_name }}" + timeout: 200 + flavor: "{{ flavor }}" + security_groups: sg-tpot-any + network: network-tpot + register: tpot + +- name: Add instance to inventory + add_host: + hostname: "{{ tpot_name.stdout }}" + ansible_host: "{{ tpot.server.public_v4 }}" + groups: tpot diff --git a/cloud/ansible/openstack/roles/deploy/vars/main.yaml b/cloud/ansible/openstack/roles/create_vm/vars/main.yaml similarity index 83% rename from cloud/ansible/openstack/roles/deploy/vars/main.yaml rename to cloud/ansible/openstack/roles/create_vm/vars/main.yaml index d2b0664a..cd56d25f 100644 --- a/cloud/ansible/openstack/roles/deploy/vars/main.yaml +++ b/cloud/ansible/openstack/roles/create_vm/vars/main.yaml @@ -2,4 +2,4 @@ availability_zone: eu-de-03 image: Standard_Debian_10_latest volume_size: 128 key_name: your-KeyPair -flavor: s2.medium.8 +flavor: s3.medium.8 From b014f73045360d00306a0f21607273f5e983417a Mon Sep 17 00:00:00 2001 From: Sebastian Haderecker Date: Sat, 13 Feb 2021 17:46:28 +0100 Subject: [PATCH 05/15] Use FQCNs --- .../openstack/roles/check/tasks/main.yaml | 6 +++--- .../openstack/roles/create_net/tasks/main.yaml | 10 +++++----- .../openstack/roles/create_vm/tasks/main.yaml | 6 +++--- .../openstack/roles/custom_ews/tasks/main.yaml | 4 ++-- .../roles/custom_hpfeeds/tasks/main.yaml | 4 ++-- .../openstack/roles/install/tasks/main.yaml | 18 +++++++++--------- .../openstack/roles/reboot/tasks/main.yaml | 4 ++-- 7 files changed, 26 insertions(+), 26 deletions(-) diff --git a/cloud/ansible/openstack/roles/check/tasks/main.yaml b/cloud/ansible/openstack/roles/check/tasks/main.yaml index fd58fc6a..3c2ac286 100644 --- a/cloud/ansible/openstack/roles/check/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/check/tasks/main.yaml @@ -1,5 +1,5 @@ - name: Install dependencies - package: + ansible.builtin.package: name: - gcc - python3-dev @@ -8,12 +8,12 @@ state: present - name: Install openstacksdk - pip: + ansible.builtin.pip: name: openstacksdk executable: pip3 - name: Check if agent forwarding is enabled - fail: + ansible.builtin.fail: msg: Please enable agent forwarding to allow Ansible to connect to the remote host! ignore_errors: yes when: lookup('env','SSH_AUTH_SOCK') == "" diff --git a/cloud/ansible/openstack/roles/create_net/tasks/main.yaml b/cloud/ansible/openstack/roles/create_net/tasks/main.yaml index 970df2d0..a8b55cd0 100644 --- a/cloud/ansible/openstack/roles/create_net/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/create_net/tasks/main.yaml @@ -1,22 +1,22 @@ - name: Create security group - os_security_group: + openstack.cloud.security_group: cloud: open-telekom-cloud name: sg-tpot-any description: tpot any-any - name: Add rules to security group - os_security_group_rule: + openstack.cloud.security_group_rule: cloud: open-telekom-cloud security_group: sg-tpot-any remote_ip_prefix: 0.0.0.0/0 - name: Create network - os_network: + openstack.cloud.network: cloud: open-telekom-cloud name: network-tpot - name: Create subnet - os_subnet: + openstack.cloud.subnet: cloud: open-telekom-cloud network_name: network-tpot name: subnet-tpot @@ -26,7 +26,7 @@ - 8.8.8.8 - name: Create router - os_router: + openstack.cloud.router: cloud: open-telekom-cloud name: router-tpot interfaces: diff --git a/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml b/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml index c04be549..c6cf3a2d 100644 --- a/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml @@ -1,9 +1,9 @@ - name: Generate T-Pot name - set_fact: + ansible.builtin.set_fact: tpot_name: "t-pot-ansible-{{ lookup('password', '/dev/null chars=ascii_lowercase,digits length=6') }}" - name: Create instance - os_server: + openstack.cloud.server: cloud: open-telekom-cloud name: "{{ tpot_name.stdout }}" availability_zone: "{{ availability_zone }}" @@ -18,7 +18,7 @@ register: tpot - name: Add instance to inventory - add_host: + ansible.builtin.add_host: hostname: "{{ tpot_name.stdout }}" ansible_host: "{{ tpot.server.public_v4 }}" groups: tpot diff --git a/cloud/ansible/openstack/roles/custom_ews/tasks/main.yaml b/cloud/ansible/openstack/roles/custom_ews/tasks/main.yaml index 1856a3df..fec93410 100644 --- a/cloud/ansible/openstack/roles/custom_ews/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/custom_ews/tasks/main.yaml @@ -1,5 +1,5 @@ - name: Copy ews configuration file - template: + ansible.builtin.template: src: ews.cfg dest: /data/ews/conf owner: root @@ -7,7 +7,7 @@ mode: 0644 - name: Patching tpot.yml with custom ews configuration file - lineinfile: + ansible.builtin.lineinfile: path: /opt/tpot/etc/tpot.yml insertafter: "/opt/ewsposter/ews.ip" line: " - /data/ews/conf/ews.cfg:/opt/ewsposter/ews.cfg" diff --git a/cloud/ansible/openstack/roles/custom_hpfeeds/tasks/main.yaml b/cloud/ansible/openstack/roles/custom_hpfeeds/tasks/main.yaml index 50ea7311..fa479137 100644 --- a/cloud/ansible/openstack/roles/custom_hpfeeds/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/custom_hpfeeds/tasks/main.yaml @@ -1,5 +1,5 @@ - name: Copy hpfeeds configuration file - copy: + ansible.builtin.copy: src: hpfeeds.cfg dest: /data/ews/conf owner: tpot @@ -8,5 +8,5 @@ register: config - name: Applying hpfeeds settings - command: /opt/tpot/bin/hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg + ansible.builtin.command: /opt/tpot/bin/hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg when: config.changed == true diff --git a/cloud/ansible/openstack/roles/install/tasks/main.yaml b/cloud/ansible/openstack/roles/install/tasks/main.yaml index 173c4f08..3338294d 100644 --- a/cloud/ansible/openstack/roles/install/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/install/tasks/main.yaml @@ -1,29 +1,29 @@ - name: Waiting for SSH connection - wait_for_connection: + ansible.builtin.wait_for_connection: - name: Gathering facts - setup: + ansible.builtin.setup: - name: Cloning T-Pot install directory - git: + ansible.builtin.git: repo: "https://github.com/telekom-security/tpotce.git" dest: /root/tpot - name: Prepare to set user password - set_fact: + ansible.builtin.set_fact: user_name: "{{ ansible_user }}" user_salt: "s0mew1ck3dTpoT" no_log: true - name: Changing password for user {{ user_name }} - user: + ansible.builtin.user: name: "{{ ansible_user }}" password: "{{ user_password | password_hash('sha512', user_salt) }}" state: present shell: /bin/bash - name: Copy T-Pot configuration file - template: + ansible.builtin.template: src: ../../../../../../iso/installer/tpot.conf.dist dest: /root/tpot.conf owner: root @@ -31,15 +31,15 @@ mode: 0644 - name: Install T-Pot on instance - be patient, this might take 15 to 30 minutes depending on the connection speed. - command: /root/tpot/iso/installer/install.sh --type=auto --conf=/root/tpot.conf + ansible.builtin.command: /root/tpot/iso/installer/install.sh --type=auto --conf=/root/tpot.conf - name: Delete T-Pot configuration file - file: + ansible.builtin.file: path: /root/tpot.conf state: absent - name: Change unattended-upgrades to take default action - blockinfile: + ansible.builtin.blockinfile: dest: /etc/apt/apt.conf.d/50unattended-upgrades block: | Dpkg::Options { diff --git a/cloud/ansible/openstack/roles/reboot/tasks/main.yaml b/cloud/ansible/openstack/roles/reboot/tasks/main.yaml index 946d38d1..1d5ce5d5 100644 --- a/cloud/ansible/openstack/roles/reboot/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/reboot/tasks/main.yaml @@ -1,10 +1,10 @@ - name: Finally rebooting T-Pot - command: shutdown -r now + ansible.builtin.command: shutdown -r now async: 1 poll: 0 - name: Next login options - debug: + ansible.builtin.debug: msg: - "***** SSH Access:" - "***** ssh {{ ansible_user }}@{{ ansible_host }} -p 64295" From 780acd0384059bb4b12227b677f0b1a2d405c75b Mon Sep 17 00:00:00 2001 From: Sebastian Haderecker Date: Sat, 13 Feb 2021 17:47:48 +0100 Subject: [PATCH 06/15] Fix name --- cloud/ansible/openstack/roles/create_vm/tasks/main.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml b/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml index c6cf3a2d..9743a1e6 100644 --- a/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml @@ -5,7 +5,7 @@ - name: Create instance openstack.cloud.server: cloud: open-telekom-cloud - name: "{{ tpot_name.stdout }}" + name: "{{ tpot_name }}" availability_zone: "{{ availability_zone }}" image: "{{ image }}" boot_from_volume: yes From acc64c277107330eacb3a28ab277dbdccc14e76c Mon Sep 17 00:00:00 2001 From: Sebastian Haderecker Date: Sat, 13 Feb 2021 17:52:18 +0100 Subject: [PATCH 07/15] Fix name --- cloud/ansible/openstack/roles/create_vm/tasks/main.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml b/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml index 9743a1e6..16f9db59 100644 --- a/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml @@ -19,6 +19,6 @@ - name: Add instance to inventory ansible.builtin.add_host: - hostname: "{{ tpot_name.stdout }}" + hostname: "{{ tpot_name }}" ansible_host: "{{ tpot.server.public_v4 }}" groups: tpot From efb51f823368e62873e133faedcd314c5de4a230 Mon Sep 17 00:00:00 2001 From: Sebastian Haderecker Date: Sat, 13 Feb 2021 18:04:23 +0100 Subject: [PATCH 08/15] Add collection requirements --- cloud/ansible/openstack/requirements.yaml | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 cloud/ansible/openstack/requirements.yaml diff --git a/cloud/ansible/openstack/requirements.yaml b/cloud/ansible/openstack/requirements.yaml new file mode 100644 index 00000000..986ae0e5 --- /dev/null +++ b/cloud/ansible/openstack/requirements.yaml @@ -0,0 +1,2 @@ +collections: +- name: openstack.cloud From f6db5412933239faccdb26b9dbc7d77c44107aae Mon Sep 17 00:00:00 2001 From: Sebastian Haderecker Date: Sat, 13 Feb 2021 18:20:01 +0100 Subject: [PATCH 09/15] Update README.md --- cloud/ansible/README.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/cloud/ansible/README.md b/cloud/ansible/README.md index 05d94819..ccd44b8c 100644 --- a/cloud/ansible/README.md +++ b/cloud/ansible/README.md @@ -11,6 +11,7 @@ This example showcases the deployment on our own OpenStack based Public Cloud Of # Table of contents - [Preparation of Ansible Master](#ansible-master) - [Ansible Installation](#ansible) + - [OpenStack Collection Installation](#collection) - [Agent Forwarding](#agent-forwarding) - [Preparations in Open Telekom Cloud Console](#preparation) - [Create new project](#project) @@ -56,6 +57,11 @@ In short (if you already have Python3/pip3 installed): pip3 install ansible ``` + +## OpenStack Collection Installation +For interacting with OpenStack resources in Ansible, you need to install the collection from Ansible Galaxy: +`ansible-galaxy collection install openstack.cloud` + ## Agent Forwarding If you run the Ansible Playbook remotely on your Ansible Master Server, Agent Forwarding must be enabled in order to let Ansible connect to newly created machines. @@ -134,14 +140,14 @@ You may have to adjust the `remote_user` in the Ansible Playbook under [`opensta ## Instance settings -Located at [`openstack/roles/deploy/vars/main.yaml`](openstack/roles/deploy/vars/main.yaml). +Located at [`openstack/roles/create_vm/vars/main.yaml`](openstack/roles/create_vm/vars/main.yaml). Here you can customize your virtual machine specifications: - Choose an availability zone. For Open Telekom Cloud reference see [here](https://docs.otc.t-systems.com/en-us/endpoint/index.html). - Change the OS image (For T-Pot we need Debian) - (Optional) Change the volume size - Specify your key pair (:warning: Mandatory) - (Optional) Change the instance type (flavor) - `s2.medium.8` corresponds to 1 vCPU and 8GB of RAM and is the minimum required flavor. + `s3.medium.8` corresponds to 1 vCPU and 8GB of RAM and is the minimum required flavor. A full list of Open Telekom Cloud flavors can be found [here](https://docs.otc.t-systems.com/en-us/usermanual/ecs/en-us_topic_0177512565.html). ``` @@ -149,7 +155,7 @@ availability_zone: eu-de-03 image: Standard_Debian_10_latest volume_size: 128 key_name: your-KeyPair -flavor: s2.medium.8 +flavor: s3.medium.8 ``` @@ -200,7 +206,7 @@ Enable this by uncommenting the role in the [deploy_tpot.yaml](openstack/deploy_ # - custom_hpfeeds ``` -You can specify custom HPFEEDS in [`openstack/roles/custom_hpfeeds/templates/hpfeeds.cfg`](openstack/roles/custom_hpfeeds/templates/hpfeeds.cfg). +You can specify custom HPFEEDS in [`openstack/roles/custom_hpfeeds/files/hpfeeds.cfg`](openstack/roles/custom_hpfeeds/files/hpfeeds.cfg). That file contains the defaults (turned off) and you can adapt it for your needs, e.g. for SISSDEN: ``` myENABLE=true From 423914f63f7844d663f8729e247aee6eb0624834 Mon Sep 17 00:00:00 2001 From: Sebastian Haderecker Date: Sat, 13 Feb 2021 18:29:27 +0100 Subject: [PATCH 10/15] Unify cloud parameter --- cloud/ansible/openstack/deploy_tpot.yaml | 1 + cloud/ansible/openstack/my_os_cloud.yaml | 2 ++ .../ansible/openstack/roles/create_net/tasks/main.yaml | 10 +++++----- .../ansible/openstack/roles/create_vm/tasks/main.yaml | 2 +- 4 files changed, 9 insertions(+), 6 deletions(-) create mode 100644 cloud/ansible/openstack/my_os_cloud.yaml diff --git a/cloud/ansible/openstack/deploy_tpot.yaml b/cloud/ansible/openstack/deploy_tpot.yaml index 9bcc22a8..02616d5c 100644 --- a/cloud/ansible/openstack/deploy_tpot.yaml +++ b/cloud/ansible/openstack/deploy_tpot.yaml @@ -6,6 +6,7 @@ - name: Deploy instance hosts: localhost + vars_files: my_os_cloud.yaml roles: - create_net - create_vm diff --git a/cloud/ansible/openstack/my_os_cloud.yaml b/cloud/ansible/openstack/my_os_cloud.yaml new file mode 100644 index 00000000..d3832f85 --- /dev/null +++ b/cloud/ansible/openstack/my_os_cloud.yaml @@ -0,0 +1,2 @@ +# Enter the name of your cloud to use from clouds.yaml +cloud: open-telekom-cloud diff --git a/cloud/ansible/openstack/roles/create_net/tasks/main.yaml b/cloud/ansible/openstack/roles/create_net/tasks/main.yaml index a8b55cd0..c372d8bb 100644 --- a/cloud/ansible/openstack/roles/create_net/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/create_net/tasks/main.yaml @@ -1,23 +1,23 @@ - name: Create security group openstack.cloud.security_group: - cloud: open-telekom-cloud + cloud: "{{ cloud }}" name: sg-tpot-any description: tpot any-any - name: Add rules to security group openstack.cloud.security_group_rule: - cloud: open-telekom-cloud + cloud: "{{ cloud }}" security_group: sg-tpot-any remote_ip_prefix: 0.0.0.0/0 - name: Create network openstack.cloud.network: - cloud: open-telekom-cloud + cloud: "{{ cloud }}" name: network-tpot - name: Create subnet openstack.cloud.subnet: - cloud: open-telekom-cloud + cloud: "{{ cloud }}" network_name: network-tpot name: subnet-tpot cidr: 192.168.0.0/24 @@ -27,7 +27,7 @@ - name: Create router openstack.cloud.router: - cloud: open-telekom-cloud + cloud: "{{ cloud }}" name: router-tpot interfaces: - subnet-tpot diff --git a/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml b/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml index 16f9db59..a9e6762e 100644 --- a/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml @@ -4,7 +4,7 @@ - name: Create instance openstack.cloud.server: - cloud: open-telekom-cloud + cloud: "{{ cloud }}" name: "{{ tpot_name }}" availability_zone: "{{ availability_zone }}" image: "{{ image }}" From ca4946c87c1176da83f3c7e4e0f980cff9f6e501 Mon Sep 17 00:00:00 2001 From: Sebastian Haderecker Date: Sat, 13 Feb 2021 18:58:42 +0100 Subject: [PATCH 11/15] Update gitignore --- cloud/.gitignore | 10 ++++++++++ cloud/ansible/.gitignore | 2 -- cloud/terraform/.gitignore | 2 -- 3 files changed, 10 insertions(+), 4 deletions(-) create mode 100644 cloud/.gitignore delete mode 100644 cloud/ansible/.gitignore delete mode 100644 cloud/terraform/.gitignore diff --git a/cloud/.gitignore b/cloud/.gitignore new file mode 100644 index 00000000..06ddc876 --- /dev/null +++ b/cloud/.gitignore @@ -0,0 +1,10 @@ +# Ansible +*.retry + +# Terraform +**/.terraform +**/terraform.* + +# OpenStack clouds +clouds.yaml +secure.yaml diff --git a/cloud/ansible/.gitignore b/cloud/ansible/.gitignore deleted file mode 100644 index 41f83492..00000000 --- a/cloud/ansible/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -# Ansible -*.retry diff --git a/cloud/terraform/.gitignore b/cloud/terraform/.gitignore deleted file mode 100644 index bcb28067..00000000 --- a/cloud/terraform/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -**/.terraform -**/terraform.* From c014e9635d857c47f323c348ff08dbffbd2e78bd Mon Sep 17 00:00:00 2001 From: Sebastian Haderecker Date: Sat, 13 Feb 2021 19:03:56 +0100 Subject: [PATCH 12/15] Update README.md --- cloud/ansible/README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/cloud/ansible/README.md b/cloud/ansible/README.md index ccd44b8c..1b2b87ea 100644 --- a/cloud/ansible/README.md +++ b/cloud/ansible/README.md @@ -132,7 +132,13 @@ clouds: user_domain_name: OTC-EU-DE-000000000010000XXXXX ``` You can also perform different authentication methods like sourcing OpenStack OS_* environment variables or providing an inline dictionary. -For more information have a look in the [os_server](https://docs.ansible.com/ansible/latest/modules/os_server_module.html) Ansible module documentation. +For more information have a look in the [openstack.cloud.server](https://docs.ansible.com/ansible/latest/collections/openstack/cloud/server_module.html) Ansible module documentation. + +If you already have your own `clouds.yaml` file or have multiple clouds in there, you can specify which one to use in the `openstack/my_os_cloud.yaml` file: +``` +# Enter the name of your cloud to use from clouds.yaml +cloud: open-telekom-cloud +``` ## Ansible remote user From 48e02ceb1c74077d29f96b3cd025cd6001fd0a9b Mon Sep 17 00:00:00 2001 From: Sebastian Haderecker Date: Sat, 13 Feb 2021 20:12:58 +0100 Subject: [PATCH 13/15] Allow for creation of multiple T-Pots --- cloud/ansible/openstack/deploy_tpot.yaml | 17 ++++++++++++----- .../openstack/roles/create_vm/tasks/main.yaml | 2 +- 2 files changed, 13 insertions(+), 6 deletions(-) diff --git a/cloud/ansible/openstack/deploy_tpot.yaml b/cloud/ansible/openstack/deploy_tpot.yaml index 02616d5c..5e3ee05a 100644 --- a/cloud/ansible/openstack/deploy_tpot.yaml +++ b/cloud/ansible/openstack/deploy_tpot.yaml @@ -4,14 +4,21 @@ roles: - check -- name: Deploy instance +- name: Deploy instances hosts: localhost vars_files: my_os_cloud.yaml - roles: - - create_net - - create_vm + tasks: + - name: Create security group and network + ansible.builtin.include_role: + name: create_net + - name: Create one or more instances + ansible.builtin.include_role: + name: create_vm + loop: "{{ range(0, 1) }}" + loop_control: + extended: yes -- name: Install T-Pot on new instance +- name: Install T-Pot hosts: tpot remote_user: linux become: yes diff --git a/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml b/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml index a9e6762e..71ad71e7 100644 --- a/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml +++ b/cloud/ansible/openstack/roles/create_vm/tasks/main.yaml @@ -2,7 +2,7 @@ ansible.builtin.set_fact: tpot_name: "t-pot-ansible-{{ lookup('password', '/dev/null chars=ascii_lowercase,digits length=6') }}" -- name: Create instance +- name: Create instance {{ ansible_loop.index }} of {{ ansible_loop.length }} openstack.cloud.server: cloud: "{{ cloud }}" name: "{{ tpot_name }}" From 2fb1967ef17d6227de824b7fbccde4532b1bc2ad Mon Sep 17 00:00:00 2001 From: Sebastian Haderecker Date: Sat, 13 Feb 2021 20:16:34 +0100 Subject: [PATCH 14/15] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 26ae8dfa..d5135241 100644 --- a/README.md +++ b/README.md @@ -290,7 +290,7 @@ If you would like to contribute, you can add other cloud deployments like Chef o You can find an [Ansible](https://www.ansible.com/) based T-Pot deployment in the [`cloud/ansible`](cloud/ansible) folder. The Playbook in the [`cloud/ansible/openstack`](cloud/ansible/openstack) folder is reusable for all **OpenStack** clouds out of the box. -It first creates all resources (security group, network, subnet, router), deploys a new server and then installs and configures T-Pot. +It first creates all resources (security group, network, subnet, router), deploys one (or more) new servers and then installs and configures T-Pot on them. You can have a look at the Playbook and easily adapt the deploy role for other [cloud providers](https://docs.ansible.com/ansible/latest/scenario_guides/cloud_guides.html). Check out [Ansible Galaxy](https://galaxy.ansible.com/search?keywords=&order_by=-relevance&page=1&deprecated=false&type=collection&tags=cloud) for more cloud collections. From 86cc54ee88a8848c58e0ac6102523db4e0a09d89 Mon Sep 17 00:00:00 2001 From: Sebastian Haderecker Date: Sat, 13 Feb 2021 20:39:32 +0100 Subject: [PATCH 15/15] Update README.md --- cloud/ansible/README.md | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/cloud/ansible/README.md b/cloud/ansible/README.md index 1b2b87ea..5be6a912 100644 --- a/cloud/ansible/README.md +++ b/cloud/ansible/README.md @@ -4,7 +4,7 @@ Here you can find a ready-to-use solution for your automated T-Pot deployment us It consists of an Ansible Playbook with multiple roles, which is reusable for all [OpenStack](https://www.openstack.org/) based clouds (e.g. Open Telekom Cloud, Orange Cloud, Telefonica Open Cloud, OVH) out of the box. Apart from that you can easily adapt the deploy role to use other [cloud providers](https://docs.ansible.com/ansible/latest/scenario_guides/cloud_guides.html). Check out [Ansible Galaxy](https://galaxy.ansible.com/search?keywords=&order_by=-relevance&page=1&deprecated=false&type=collection&tags=cloud) for more cloud collections. -The Playbook first creates all resources (security group, network, subnet, router), deploys a new server and then installs and configures T-Pot. +The Playbook first creates all resources (security group, network, subnet, router), deploys one (or more) new servers and then installs and configures T-Pot on them. This example showcases the deployment on our own OpenStack based Public Cloud Offering [Open Telekom Cloud](https://open-telekom-cloud.com/en). @@ -19,8 +19,9 @@ This example showcases the deployment on our own OpenStack based Public Cloud Of - [Import Key Pair](#key-pair) - [Clone Git Repository](#clone-git) - [Settings and recommended values](#settings) - - [Clouds.yaml](#clouds-yaml) + - [clouds.yaml](#clouds-yaml) - [Ansible remote user](#remote-user) + - [Number of instances to deploy](#number) - [Instance settings](#instance-settings) - [User password](#user-password) - [Configure `tpot.conf.dist`](#tpot-conf) @@ -118,7 +119,7 @@ All Ansible related files are located in the [`cloud/ansible/openstack`](opensta You can configure all aspects of your Elastic Cloud Server and T-Pot before using the Playbook: -## Clouds.yaml +## clouds.yaml Located at [`openstack/clouds.yaml`](openstack/clouds.yaml). Enter your Open Telekom Cloud API user credentials here (username, password, project name, user domain name): ``` @@ -144,6 +145,14 @@ cloud: open-telekom-cloud ## Ansible remote user You may have to adjust the `remote_user` in the Ansible Playbook under [`openstack/deploy_tpot.yaml`](openstack/deploy_tpot.yaml) depending on your Debian base image (e.g. on Open Telekom Cloud the default Debian user is `linux`). + +## Number of instances to deploy +You can adjust the number of VMs/T-Pots that you want to create in [`openstack/deploy_tpot.yaml`](openstack/deploy_tpot.yaml): +``` +loop: "{{ range(0, 1) }}" +``` +One instance is set as the default, increase to your liking. + ## Instance settings Located at [`openstack/roles/create_vm/vars/main.yaml`](openstack/roles/create_vm/vars/main.yaml). @@ -228,6 +237,7 @@ myFORMAT=json # Deploying a T-Pot :honey_pot::honeybee: Now, after configuring everything, we can finally start deploying T-Pots! + Go to the [`openstack`](openstack) folder and run the Ansible Playbook with: `ansible-playbook deploy_tpot.yaml` (Yes, it is as easy as that :smile:) @@ -235,8 +245,8 @@ Go to the [`openstack`](openstack) folder and run the Ansible Playbook with: If you are running on a machine which asks for a sudo password, you can use: `ansible-playbook --ask-become-pass deploy_tpot.yaml` -The Playbook will first install required packages on the Ansible Master and then deploy a new server instance. -After that, T-Pot gets installed and configured on the newly created host, optionally custom configs are applied and finally it reboots. +The Playbook will first install required packages on the Ansible Master and then deploy one (or more) new server instances. +After that, T-Pot gets installed and configured on them, optionally custom configs are applied and finally it reboots. Once this is done, you can proceed with connecting/logging in to the T-Pot according to the [documentation](https://github.com/telekom-security/tpotce#ssh-and-web-access).