diff --git a/CHANGELOG.md b/CHANGELOG.md
index 68a48f08..0bfabca1 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,11 @@
# Changelog
+## 20190626
+- **HPFEEDS Opt-In commandline option**
+ - Pass a hpfeeds config file as a commandline argument
+ - hpfeeds config is saved in `/data/ews/conf/hpfeeds.cfg`
+ - Update script restores hpfeeds config
+
## 20190604
- **Finalize Fatt support**
- Build visualizations, searches, dashboards
diff --git a/README.md b/README.md
index 83846c0d..e2ec6757 100644
--- a/README.md
+++ b/README.md
@@ -122,6 +122,10 @@ h apt-fast the downloads will be even faster by downloading packages not only in
- **Added Ansible T-Pot Deployment on Open Telekom Cloud**
- Reusable Ansible Playbooks for all cloud providers
- Example Showcase with our Open Telekom Cloud
+- **HPFEEDS Opt-In commandline option**
+ - Pass a hpfeeds config file as a commandline argument
+ - hpfeeds config is saved in `/data/ews/conf/hpfeeds.cfg`
+ - Update script restores hpfeeds config
# Technical Concept
@@ -375,11 +379,13 @@ T-Pot requires outgoing git, http, https connections for updates (Debian, Docker
# Updates
For the ones of you who want to live on the bleeding edge of T-Pot development we introduced an update feature which will allow you to update all T-Pot relevant files to be up to date with the T-Pot master branch.
**If you made any relevant changes to the T-Pot relevant config files make sure to create a backup first.**
+
The Update script will:
- - **merciless** overwrite local changes to be in sync with the T-Pot master branch
+ - **mercilessly** overwrite local changes to be in sync with the T-Pot master branch
- upgrade the system to the packages available in Debian (Sid)
- update all resources to be in-sync with the T-Pot master branch
- ensure all T-Pot relevant system files will be patched / copied into the original T-Pot state
+ - restore your custom ews.cfg and HPFEED settings from `/data/ews/conf`
You simply run the update script:
```
@@ -467,8 +473,17 @@ We encourage you not to disable the data submission as it is the main purpose of
## Opt-In HPFEEDS Data Submission
-As an Opt-In it is now possible to also share T-Pot data with 3rd party HPFEEDS brokers, such as [SISSDEN](https://sissden.eu).
-If you want to share your T-Pot data you simply have to register an account with a 3rd party broker with its own benefits towards the community. Once registered you will receive your credentials to share events with the broker. In T-Pot you simply run `hpfeeds_optin.sh` which will ask for your credentials, in case of SISSDEN this is just `Ident` and `Secret`, everything else is pre-configured. It will automatically update `/opt/tpot/etc/tpot.yml` to deliver events to your desired broker.
+As an Opt-In it is now possible to also share T-Pot data with 3rd party HPFEEDS brokers, such as [SISSDEN](https://sissden.eu).
+If you want to share your T-Pot data you simply have to register an account with a 3rd party broker with its own benefits towards the community. Once registered you will receive your credentials to share events with the broker. In T-Pot you simply run `hpfeeds_optin.sh` which will ask for your credentials, in case of SISSDEN this is just `Ident` and `Secret`, everything else is pre-configured.
+It will automatically update `/opt/tpot/etc/tpot.yml` to deliver events to your desired broker.
+
+The script can accept a config file as an argument, e.g. `./hpfeeds_optin.sh --conf=hpfeeds.cfg`
+
+Your current config will also be stored in `/data/ews/conf/hpfeeds.cfg` where you can review or change it.
+Be sure to apply any changes by running `./hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg`.
+No worries: You old config gets backed up in `/data/ews/conf/hpfeeds.cfg.old`
+
+Of course you can also rerun the `hpfeeds_optin.sh` script to change and apply your settings interactively.
# Roadmap
diff --git a/bin/hpfeeds_optin.sh b/bin/hpfeeds_optin.sh
index 5310e401..ff27a5b8 100755
--- a/bin/hpfeeds_optin.sh
+++ b/bin/hpfeeds_optin.sh
@@ -67,10 +67,28 @@ mySECRET="secret"
myFORMAT="json"
}
+function fuWRITETOFILE () {
+if [ -f '/data/ews/conf/hpfeeds.cfg' ]; then
+ echo "Creating backup of current config in /data/ews/conf/hpfeeds.cfg.old"
+ mv /data/ews/conf/hpfeeds.cfg /data/ews/conf/hpfeeds.cfg.old
+fi
+echo "Storing new config in /data/ews/conf/hpfeeds.cfg"
+cat >> /data/ews/conf/hpfeeds.cfg < /dev/null && [ ! -z $filename ] && [ -f $filename ]
+ then
+ source $filename
+else
+
+# Proceed with interactive setup when no config file is found
echo "HPFEEDS Delivery Opt-In for T-Pot"
echo "---------------------------------"
echo "By running this script you agree to share your data with a 3rd party and agree to their corresponding sharing terms."
@@ -120,5 +147,7 @@ while [ 1 != 2 ]
;;
esac
done
-fuAPPLY
+fi
+fuWRITETOFILE
+fuAPPLY
diff --git a/cloud/open-telekom-cloud/README.md b/cloud/open-telekom-cloud/README.md
index d43b5770..e40acdd5 100644
--- a/cloud/open-telekom-cloud/README.md
+++ b/cloud/open-telekom-cloud/README.md
@@ -18,8 +18,7 @@ This example showcases the deployment on our own Public Cloud Offering [Open Tel
- [Configure `.otc_env.sh`](#otc-env)
- [Configure `.ecs_settings.sh`](#ecs-settings)
- [Configure `tpot.conf.dist`](#tpot-conf)
- - [Optional: Custom `ews.cfg`](#ews-cfg)
- - [Optional: Configure `.hpfeeds_settings.sh`](#hpfeeds)
+ - [Optional: Custom `ews.cfg` and HPFEEDS](#ews-hpfeeds)
- [Deploying a T-Pot](#deploy)
- [Further documentation](#documentation)
@@ -171,11 +170,12 @@ myCONF_WEB_USER='webuser'
myCONF_WEB_PW='w3b$ecret'
```
-
-## Optional: Custom `ews.cfg`
-To enable this feature, set `custom_ews=true` in `.ecs_settings.sh`; See here: [Configure `.ecs_settings.sh`](#ecs-settings)
+
+## Optional: Custom `ews.cfg` and HPFEEDS
+To enable these features, set `custom_ews=true` in `.ecs_settings.sh`; See here: [Configure `.ecs_settings.sh`](#ecs-settings)
-Here you can create a custom config file for `ewsposter`.
+### ews.cfg
+You can use a custom config file for `ewsposter`.
e.g. when you have your own credentials for delivering data to our [Sicherheitstacho](https://sicherheitstacho.eu/start/main).
You can find the `ews.cfg` template file here: [`ansible/roles/custom_ews/templates/ews.cfg`](ansible/roles/custom_ews/templates/ews.cfg) and adapt it for your needs.
@@ -193,21 +193,21 @@ token = your_token
...
```
-
-## Optional: Configure `.hpfeeds_settings.sh`
-When the `custom_ews=true` setting is set in`.ecs_settings.sh` (see here: [Configure `.ecs_settings.sh`](#ecs-settings)), you can also specify custom HPFEEDS in `.hpfeeds_settings.sh`.
-e.g. for SISSDEN:
+### HPFEEDS
+You can also specify HPFEEDS in [`ansible/roles/custom_ews/templates/hpfeeds.cfg`](ansible/roles/custom_ews/templates/hpfeeds.cfg).
+That file constains the defaults (turned off) and you can adapt it for your needs, e.g. for SISSDEN:
```
myENABLE=true
myHOST=hpfeeds.sissden.eu
myPORT=10000
myCHANNEL=t-pot.events
+myCERT=/opt/ewsposter/sissden.pem
myIDENT=your_user
mySECRET=your_secret
-myCERT=/opt/ewsposter/sissden.pem
myFORMAT=json
```
+
# Deploying a T-Pot :honey_pot::honeybee:
Now, after configuring everything, we can finally start deploying T-Pots:
diff --git a/cloud/open-telekom-cloud/ansible/roles/custom_ews/tasks/main.yaml b/cloud/open-telekom-cloud/ansible/roles/custom_ews/tasks/main.yaml
index 06404182..b49d4df4 100644
--- a/cloud/open-telekom-cloud/ansible/roles/custom_ews/tasks/main.yaml
+++ b/cloud/open-telekom-cloud/ansible/roles/custom_ews/tasks/main.yaml
@@ -12,29 +12,13 @@
insertafter: '/opt/ewsposter/ews.ip'
line: ' - /data/ews/conf/ews.cfg:/opt/ewsposter/ews.cfg'
-- name: Lookup HPFEED environment variables
- set_fact:
- myENABLE: "{{ lookup('env', 'myENABLE') }}"
- myHOST: "{{ lookup('env', 'myHOST') }}"
- myPORT: "{{ lookup('env', 'myPORT') }}"
- myCHANNEL: "{{ lookup('env', 'myCHANNEL') }}"
- myIDENT: "{{ lookup('env', 'myIDENT') }}"
- mySECRET: "{{ lookup('env', 'mySECRET') }}"
- myCERT: "{{ lookup('env', 'myCERT') }}"
- myFORMAT: "{{ lookup('env', 'myFORMAT') }}"
+- name: Copy hpfeeds configuration file
+ template:
+ src: ../templates/hpfeeds.cfg
+ dest: /data/ews/conf
+ owner: root
+ group: root
+ mode: 0644
-- name: Apply HPFEED settings in tpot.yml
- lineinfile:
- path: /opt/tpot/etc/tpot.yml
- regexp: "{{ item.regexp }}"
- line: "{{ item.line }}"
- with_items:
- - { regexp: 'EWS_HPFEEDS_ENABLE.*', line: ' - EWS_HPFEEDS_ENABLE={{ myENABLE | lower }}' }
- - { regexp: 'EWS_HPFEEDS_HOST.*', line: ' - EWS_HPFEEDS_HOST={{ myHOST }}' }
- - { regexp: 'EWS_HPFEEDS_PORT.*', line: ' - EWS_HPFEEDS_PORT={{ myPORT }}' }
- - { regexp: 'EWS_HPFEEDS_CHANNELS.*', line: ' - EWS_HPFEEDS_CHANNELS={{ myCHANNEL }}' }
- - { regexp: 'EWS_HPFEEDS_IDENT.*', line: ' - EWS_HPFEEDS_IDENT={{ myIDENT }}' }
- - { regexp: 'EWS_HPFEEDS_SECRET.*', line: ' - EWS_HPFEEDS_SECRET={{ mySECRET }}' }
- - { regexp: 'EWS_HPFEEDS_TLSCERT.*', line: ' - EWS_HPFEEDS_TLSCERT={{ myCERT }}' }
- - { regexp: 'EWS_HPFEEDS_TLSCERT.False', line: ' - EWS_HPFEEDS_TLSCERT={{ myCERT | lower }}' }
- - { regexp: 'EWS_HPFEEDS_FORMAT.*', line: ' - EWS_HPFEEDS_FORMAT={{ myFORMAT }}' }
+- name: Applying hpfeeds settings
+ command: /opt/tpot/bin/hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg
diff --git a/cloud/open-telekom-cloud/.hpfeeds_settings.sh b/cloud/open-telekom-cloud/ansible/roles/custom_ews/templates/hpfeeds.cfg
similarity index 100%
rename from cloud/open-telekom-cloud/.hpfeeds_settings.sh
rename to cloud/open-telekom-cloud/ansible/roles/custom_ews/templates/hpfeeds.cfg
index aae1dcad..1b981fdd 100644
--- a/cloud/open-telekom-cloud/.hpfeeds_settings.sh
+++ b/cloud/open-telekom-cloud/ansible/roles/custom_ews/templates/hpfeeds.cfg
@@ -2,7 +2,7 @@ myENABLE=false
myHOST=host
myPORT=port
myCHANNEL=channels
+myCERT=false
myIDENT=user
mySECRET=secret
-myCERT=false
myFORMAT=json
diff --git a/cloud/open-telekom-cloud/deploy_ansible_otc_t-pot.sh b/cloud/open-telekom-cloud/deploy_ansible_otc_t-pot.sh
index 072ce0e1..9f3756f9 100755
--- a/cloud/open-telekom-cloud/deploy_ansible_otc_t-pot.sh
+++ b/cloud/open-telekom-cloud/deploy_ansible_otc_t-pot.sh
@@ -33,22 +33,9 @@ source .ecs_settings.sh
# Import OTC authentication credentials
source .otc_env.sh
-# Import HPFEED settings
-source .hpfeeds_settings.sh
-
# Password is later used by Ansible
export LINUX_PASS=$linuxpass
-# HPFEED settings are later used by Ansible
-export myENABLE=$myENABLE
-export myHOST=$myHOST
-export myPORT=$myPORT
-export myCHANNEL=$myCHANNEL
-export myIDENT=$myIDENT
-export mySECRET=$mySECRET
-export myCERT=$myCERT
-export myFORMAT=$myFORMAT
-
# Ignore ssh host keys as they are new anyway
export ANSIBLE_HOST_KEY_CHECKING=False
diff --git a/update.sh b/update.sh
index b8bf3814..3d2b01ce 100755
--- a/update.sh
+++ b/update.sh
@@ -253,18 +253,24 @@ echo "### https://raw.githubusercontent.com/dtag-dev-sec/tpotce/master/etc/objec
echo "### Export and import the objects easily through the Kibana WebUI:"
echo "### Go to Kibana > Management > Saved Objects > Export / Import"
echo "### All objects will be overwritten upon import, make sure to run an export first."
-echo
-echo "### Please reboot."
-echo
}
function fuRESTORE_EWSCFG () {
if [ -f '/data/ews/conf/ews.cfg' ] && ! grep 'ews.cfg' /opt/tpot/etc/tpot.yml > /dev/null; then
+ echo
echo "### Restoring volume mount for ews.cfg in tpot.yml"
sed -i '/\/opt\/ewsposter\/ews.ip/a\\ \ \ \ \ - /data/ews/conf/ews.cfg:/opt/ewsposter/ews.cfg' /opt/tpot/etc/tpot.yml
fi
}
+function fuRESTORE_HPFEEDS () {
+if [ -f '/data/ews/conf/hpfeeds.cfg' ]; then
+ echo
+ echo "### Restoring HPFEEDS in tpot.yml"
+ ./bin/hpfeeds_optin.sh --conf=/data/ews/conf/hpfeeds.cfg
+fi
+}
+
################
# Main section #
@@ -297,3 +303,8 @@ fuBACKUP
fuSELFUPDATE "$0" "$@"
fuUPDATER
fuRESTORE_EWSCFG
+fuRESTORE_HPFEEDS
+
+echo
+echo "### Please reboot."
+echo