0

Ansible – Using template lookup

For some of actions / modules / roles it might be that you would like to use your Jinja template as variable. I have been looking some time for this lookup module. Therefore to save you from crawling over internet this is how could you use it

Example of Jinja template

  location ~ \.php$ {
    try_files $uri =404;
    root           /var/www/htdocs/;
    fastcgi_pass   unix:/var/run/php65-php-fpm.sock;
    fastcgi_intercept_errors        on;
    fastcgi_buffers 256 16k;
    fastcgi_max_temp_file_size 0;
    fastcgi_index  index.php;
    fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include        fastcgi_params;

    {% if env is defined and env=="testos" %}
    auth_basic           "Ninja Test ";
    auth_basic_user_file {{ nginx_basic_auth }} ;
    {% endif %}
  }

And then within our playbook we define it as following

---
- name: Testing
  hosts: localhost
  vars:
    - blabla: "Testing123"
    - testing: "{{ lookup('template', 'template.j2') }}"

  tasks:
  - debug: msg="{{testing}}"

 

Are you using this in other way ? Comment whats your approach 🙂

0

Ansible – Using dictionary to deploy pem certificates

When automating certificate deployments I wanted to have smart way of deploying them. So I went ahead and decided to use dictionaries.

For this example my variables looked more like as following :

ssl_certificates:
    domain_uno_com:
      owner: haproxy
      group: haproxy
      mode: "u=r,go="
      certificate: |
                                              -----BEGIN CERTIFICATE-----
                                              dslkajfafak234h23o4h32jkh43jqtghkjafhads;fhd89fuad9f6a8s7f6adsf
                                              < ..................... bogus info uno ...................... >
                                              yjEdslkajfafak234h23o4h32jkh43jZlcmlTaWduLCBJbmMuMR8wHQYDVQQL23
                                                -----END CERTIFICATE-----
      key: |
                                              -----BEGIN PRIVATE KEY-----
                                              dslkajfafak234h23o4h32jkh43jqtghkjafhads;fhd89fuad9f6a8s7f6adsf
                                              < ..................... bogus info uno ...................... >
                                              yjEdslkajfafak234h23o4h32jkh43jZlcmlTaWduLCBJbmMuMR8wHQYDVQQL23
                                              Edslkajfafak234h==
                                              -----END PRIVATE KEY-----
      
    domain_duo_com:
      owner: haproxy
      group: haproxy
      mode: "u=r,go="
      certificate: |
                                              -----BEGIN CERTIFICATE-----
                                              dslkajfafak234h23o4h32jkh43jqtghkjafhads;fhd89fuad9f6a8s7f6adsf
                                              < ..................... bogus info duo ...................... >
                                              yjEdslkajfafak234h23o4h32jkh43jZlcmlTaWduLCBJbmMuMR8wHQYDVQQL23
                                                -----END CERTIFICATE-----
      key: |
                                              -----BEGIN PRIVATE KEY-----
                                              dslkajfafak234h23o4h32jkh43jqtghkjafhads;fhd89fuad9f6a8s7f6adsf
                                              < ..................... bogus info duo ...................... >
                                              yjEdslkajfafak234h23o4h32jkh43jZlcmlTaWduLCBJbmMuMR8wHQYDVQQL23
                                              Edslkajfafak234h==
                                              -----END PRIVATE KEY-----

 

Once we have that within our playbook we will be using the following actions to create ourselves pem files

       - name: SSL certificates Web | Create certificate key files
         copy:
           dest: "{{web_ssl_folder}}/{{ item.key.replace('_','.') }}.pem"
           content: "{{ item.value.certificate + '\n' + item.value.key }}"
           owner: "{{ item.value.owner }}"
           group: "{{ item.value.group }}"
           mode: "{{ item.value.mode }}"
         with_dict: ssl_certificates
         no_log: true

 

Now when we run our playbook what will happen is we will get within folder defined under web_ssl_folder  new certificates called respectively domain.uno.com.pem and domain.duo.com.pem.

Of course if you add more entries you will get more created. So for you the only thing to change from here is the owner and possibly the rights ( although think twice 🙂 )

9

Vyos – Site to site VPN using VTI and OSPF

Hey! So today we will be challenging setup of vyos site-to-site VPN. In theory there is nothing really difficult about that one – its just choosing the right options.

What I would really would like to highlight here which I believe is quite useful ( although not always possible to achieve ) is to use VTI ( Virtual Tunnel Interface ) instead of local/remote prefixes ( more reading here ). So let’s get to do it 🙂 All of these commands will be part of single commit.

First we tackle the virtual interface

set interfaces vti vti0 address 10.1.0.1/32
set interfaces vti vti0 description 'VPN_VTI0'

Of course description is optional however I like the system to be well described so other people can also benefit from understanding quickly what and why is it.

Next I fill out phase 1 and phase 2 settings. And here I cannot help much as it will differ per endpoint you are setting your vpn with. So treat this only as point of reference

# Phase 2 
set vpn ipsec esp-group ESP-Default compression 'disable' 
set vpn ipsec esp-group ESP-Default lifetime '3600' 
set vpn ipsec esp-group ESP-Default mode 'tunnel' 
set vpn ipsec esp-group ESP-Default pfs 'dh-group2' 
set vpn ipsec esp-group ESP-Default proposal 1 encryption '3des' 
set vpn ipsec esp-group ESP-Default proposal 1 hash 'sha1' 

# Phase 1 
set vpn ipsec ike-group IKE-Default dead-peer-detection action 'clear' 
set vpn ipsec ike-group IKE-Default dead-peer-detection interval '30' 
set vpn ipsec ike-group IKE-Default dead-peer-detection timeout '90' 
set vpn ipsec ike-group IKE-Default ikev2-reauth 'no' 
set vpn ipsec ike-group IKE-Default key-exchange 'ikev1' 
set vpn ipsec ike-group IKE-Default lifetime '86400' 
set vpn ipsec ike-group IKE-Default proposal 1 dh-group '2' 
set vpn ipsec ike-group IKE-Default proposal 1 encryption 'aes256' 
set vpn ipsec ike-group IKE-Default proposal 1 hash 'sha256'

 

Cool – so we got the base – now we need to define which interface will handle our VPN traffic

set vpn ipsec ipsec-interfaces interface 'eth1' 
set vpn ipsec logging log-modes 'all'

 

And then I move to configuring site-to-site vpn

# Setup the site-2-site config
set vpn ipsec site-to-site peer <remote-IP-address> authentication id '<local-WAN-IP-address>' 
set vpn ipsec site-to-site peer <remote-IP-address> authentication mode 'pre-shared-secret' 
set vpn ipsec site-to-site peer <remote-IP-address> authentication pre-shared-secret 'this-Of-course-Is-Some-paaaassshpraseeee' 
set vpn ipsec site-to-site peer <remote-IP-address> connection-type 'initiate' 
set vpn ipsec site-to-site peer <remote-IP-address> default-esp-group 'ESP-Default' 
set vpn ipsec site-to-site peer <remote-IP-address> ike-group 'IKE-Default' 
set vpn ipsec site-to-site peer <remote-IP-address> ikev2-reauth 'inherit' 
set vpn ipsec site-to-site peer <remote-IP-address> local-address '<local-WAN-IP-address>' 

Once done we need to associate our VTI interface with this site to site VPN. This is done by issuing the following

# Make use of our VTI interface
set vpn ipsec site-to-site peer <remote-IP-address> vti bind vti1
set vpn ipsec site-to-site peer <remote-IP-address> vti esp-group ESP-Default

 

Cool – easy wasn’t ? 🙂 Now we can configure OSPF. I did mine this way

set protocols ospf parameters router-id <remote-IP-address>
set protocols ospf area 0.0.0.0 network 192.168.1.0/24
set protocols ospf area 0.0.0.0 network 192.168.2.0/24
set interfaces vti vti0 ip ospf network point-to-point

 

Full config as usual available in github

 

 

Any experience on challenges ? Maybe better ways ? Post in comments!

0

HAproxy – securing Piwik stats

So today we will be discussing how to secure Piwik stats with help of HAproxy. I didn’t find direct articles relating to how to do that so google and quick tests were my companions for this 🙂

So Piwik is quite cool project for getting your statistics. I have created a simple docker container out of it ( available on DockerHub ) and now run it on my server. Looking at documentation I wanted to have it secured so I have found the following quote :

” Use .htaccess to restrict access to a few files only, and to restrict by defined IP address.
For those running an Apache web server, it should be easy to use .htaccess (Apache’s Access Control List) files to restrict entry to Piwik. For example, when you restrict access to files, you will need to allow external access to the piwik.php and piwik.js files, as well as to the URL index.php?module=CoreAdminHome&action=optOut (for the opt-out iframe).”

 

So what I have done – was creating simple ACL for piwik ( yep – you can have the same name of ACLs 🙂 ) called piwik_stats and used it when deciding for backend so backend would be accessible only if coming from whitelisted IP address or only when using whitelisted parts of URL 🙂

    # Whitelist : MyHome
    acl net_whitelist_myhome  src -f /etc/haproxy/myhome

    # Piwik stats security
    acl piwik_stats  url  /index.php?module=CoreAdminHome&action=optOut
    acl piwik_stats  path /piwik.php
    acl piwik_stats  path /piwik.js

    use_backend stats_piwik  if domain_stats.example_com piwik_stats or net_whitelist_myhome

 

Now I know there are probably better ways to do that- if you have one – share your comments / thoughts.

 

Enjoy 🙂

 

 

6

HAproxy – Logging within docker container

Hey ,

So today we will continue looking at HAproxy – however this time we will be using Docker to host our load balancer. As far as it is no problem to just download the main image from docker hub and run instantly it does not give out of the box something that I was after …… the logs.

That’s why I went ahead and just created my own version of HAproxy which includes rsyslog. Repository with the image can be found on github.

In order to run the container we just need to execute the following commands:

  1. To get the most up to date image from my docker hub repo
    docker pull rafpe/docker-haproxy-rsyslog
  2. To start container ( assuming you have config file in current directory )
    docker run -it -d -P -v ${PWD}/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg rafpe/docker-haproxy-rsyslog

 

Once you do this container should be up and running and if you query for current docker containers you should see something similar to output below :

haproxy_rsyslog_logs

 

As you can see we are getting logs directly visible after querying with docker logs command.

In one of the future posts we will be investigating logs format customisations as well as features included in HAproxy since 1.6  which is log tags.

 

If you would have any problems configuring this because of missing config you can use sample below

global
    log 127.0.0.1 local2
    maxconn 2000
    pidfile /var/run/haproxy.pid

    tune.ssl.default-dh-param 2048

    # SSL ciphers
    ssl-default-bind-options no-sslv3 no-tls-tickets
    ssl-default-bind-ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA



defaults
    mode    http
    option  httplog
    option  dontlognull
    option  forwardfor
    option  contstats
    option  http-server-close
    option log-health-checks
    retries 3
    option  redispatch
    timeout connect  5000
    timeout client  10000
    timeout server  10000

    # make sure log-format is on a single line
    log global
    log-format {"type":"haproxy","timestamp":%Ts,"http_status":%ST,"http_request":"%r","remote_addr":"%ci","bytes_read":%B,"upstream_addr":"%si","backend_name":"%b","retries":%rc,"bytes_uploaded":%U,"upstream_response_time":"%Tr","upstream_connect_time":"%Tc","session_duration":"%Tt","termination_state":"%ts"}


frontend http-in
    bind *:80
    
    # Default backend to be used
    default_backend will-be-back-soon


backend will-be-back-soon
   balance roundrobin

 

1

HAproxy – ACL on X-Forwarded-For

HeyYo,

So today we will work on having correct ACLs but when we are behind a service i.e. Akamai or Cloudflare . Benefits of those services are beyond scope of this post however if you are into CDN cache / security / dDos protection then take a look.

Challenge which we are looking into today is to have appropriate ACLs within our configuration. Usually simple ACL could look like :

	  # Check restricted network
	  acl restricted_network src 90.80.70.60 # Home network

 

and this is perfectly valid configuration entry which will work if the client is hitting our server directly. However if we decide to use one of mentioned above solutions ( or any other ) we need to make sure we can still apply our ACLs as source ip will be different. Well no worries – HAproxy is such a versatile load balancer that this is so easy to achieve. We will use here a header being passed from our service provider hiding under X-Forwarded-For which will contain client original IP address

So what I did first was to create a file ( name is arbitrary for demo ) called acl_restricted_network and placed it under /etc/haproxy/acl_restricted_network

Content of this file are IP addresses / networks which I will use in my ACL list and looks as simple as :

1.2.3.4/32
5.6.7.8/32

 

Then the last thing to change in our configuration of haproxy is to have this file being checked on desired ACL

    acl restricted_network hdr_ip(X-Forwarded-For) -f /etc/haproxy/acl_restricted_network

 

Now to ease of those of you that are worried about performance here is the quote from one of mailing lists detailing this approach

IP lists loaded from files are stored in binary trees. Even if you load one million prefixes, you should barely notice it under load, as the prefix lookup is cheaper than the header extraction itself.

 

And voilla 🙂 it works out of hand without any problems 🙂 and is performance friendly !

 

And if you are looking for more posts about haproxy checkout articles below:

[display-posts category=”haproxy” posts_per_page=”-1″ include_date=”true” order=”ASC” orderby=”title”]

 

11

Docker compose v2 – using static network addresses

Docker compose is a really great piece of code 🙂 that will allow you to build better orchestration with your containers. Recent breaking releases introduced a lot of features. While looking at some of them I was wondering about situations in which you build more (or a bit less ) complex containers based environment and do not have service discovery. In some instances you would just like to have static IP addresses.

Now this is perfectly easy to be done when running containers with cli … but how do you do that with compose ? After looking at the documentation I managed to come out with the following

And this is allowing me to specify static IP addresses for my containers using the compose file. For reference you can find the snippet of full file below

version: '2'

services:
  haproxy:
       image: haproxy:latest
       ports:
          - "80:80"
          - "443:443"
       volumes:
          - ${PWD}/haproxy/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg
       restart: always
       networks:
          - widgets
       logging:
        driver: json-file
        options:
          max-size: "100m"
          max-file: "3"
          labels: "haproxy"

  mariadb:
       image: mariadb:latest
       volumes:
          - /vol/appdata/mariadb:/var/lib/mysql
       environment:
          - MYSQL_ROOT_PASSWORD=secret-pw
       restart: always
       networks:
          - widgets
       logging:
         driver: json-file
         options:
           max-size: "100m"
           max-file: "3"
           labels: "mariadb"

  app_orangella:
       image: apache:1.0
       restart: always
       ports:
          - "81:80"
       networks:
          - widgets
       logging:
         driver: json-file
         options:
           max-size: "50m"
           max-file: "3"
           labels: "app_orangella"

networks:
  widgets:
    driver: bridge
    ipam:
     config:
       - subnet: 172.10.0.0/16
         gateway: 172.10.5.254
         aux_addresses:
          haproxy: 172.10.1.2
          mariadb: 172.10.1.3
          app_orangella: 172.10.1.4

 

hope this will get you rolling with Docker compose 🙂

 

0

HAproxy – SSL domains in crt-list

I think for those using high throughput to load balancers will know HAproxy immediately. If HAproxy is something new to you – I highly recommend to scatter around and get your self familiar with this great product. I use it personally and as well recommend it ( if the requirements match ) to my customers. I thought I will create separate category especially for this awesome piece of art and will share with you some of my challenges and discoveries I came across with.

So today I will start with the fact that HAproxy supports SNI and that you can have multiple certificates assigned. If you look at internet ( or even at the documentation ) you will see its common to use syntax like :

frontend https-in
    bind *:443 ssl crt /etc/ssl/server1.pem crt /etc/ssl/server2.pem
    
    http-request set-header X-Forwarded-Proto https 
    
    default_backend application-backend

 

What you can see here is that we are specifying certificates ( detailed way of HApoxy handles this can be found under link ) .  However I have been recently using crt-list which allows me to specify certificates for domains ( and also do filtering within that file ) .

File looks as easy as (basic no filtering ) :

/etc/ssl/web/domain1.net.pem domain1.net
/etc/ssl/web/domain2.net.pem domain2.net

 

From there in my config I use the following :

#  _____                    _                    _
# |  ___|_ __  ___   _ __  | |_  ___  _ __    __| | ___
# | |_  | '__|/ _ \ | '_ \ | __|/ _ \| '_ \  / _` |/ __|
# |  _| | |  | (_) || | | || |_|  __/| | | || (_| |\__ \
# |_|   |_|   \___/ |_| |_| \__|\___||_| |_| \__,_||___/


frontend http-in
    bind 0.0.0.0:80
    redirect scheme https code 301 if !{ ssl_fc }


frontend https-in
    bind 0.0.0.0:443 ssl crt-list /etc/haproxy/crt-list.txt

    http-request set-header X-Forwarded-Proto https if { ssl_fc }

 

And thats how easy it is. In coming posts I will try to publish more interesting information abut HAproxy.

0

MySql SSL – require client certificate for user

When working with MySql database where you have setup encryption following one of many guides on internet you then have choice between just requires SSL to be used or that the client also has certificate. I followed the complete guide from mysql dev which allowed me to quickly get the certificates and SSL setup for my database.

Then depending on your choice you can create users using snippets below :