0

Compiling NGINX with — with-http_auth_request_module on Centos 7

While looking at SSO solutions I decided to investigate a bit more options how I could use Nginx with solutions like Okta to protect my resources. One of interesting ones was using authentication proxy with Nginx.

The afore functionality is available through use of http_auth_request_module. However this module is not compiled by default. This got me the idea that would be nice to exercise going step by step through compiling Nginx with auth module Centos 7.

Yes – I do know that there are solutions on the market/internet which would save me from this – however I value the learning process in this challenge as well πŸ™‚ If you have interesting links to alternatives please leave them in the comment section.

Getting the sources

Our journey begins with getting the sources. I have tried following the official Nginx documentation but I find it …. somehow not up to the task. Hence there are some modifications or additions that I did to get this through πŸ™‚

mkdir nginx-from-source && cd $_

Once we have our new folder we can download the pre-reqs

Here we are taking Nginx version 1.19.0 – please be sure to check whats the latest version before running the command

   wget https://ftp.pcre.org/pub/pcre/pcre-8.44.tar.gz
   wget http://zlib.net/zlib-1.2.11.tar.gz
   wget http://www.openssl.org/source/openssl-1.1.1g.tar.gz
   wget https://nginx.org/download/nginx-1.19.0.tar.gz
   tar zxf nginx-1.19.0.tar.gz

Compile PCRE

tar -zxf pcre-8.44.tar.gz
cd pcre-8.44
./configure
make
sudo make install

Compile ZLIB

tar -zxf zlib-1.2.11.tar.gz
cd zlib-1.2.11
./configure
make
sudo make install

Compiling OpenSSL

OpenSSL deserves spot for bit more insights than just dry code. We will use never version than the one running on the box right now.

Pre-reqs

We will start off by installing required packages via yum and extracting the content of downloaded archive

yum group install 'Development Tools'
yum install perl-core zlib-devel -y
tar -xf openssl-1.1.1g.tar.gz 
cd openssl-1.1.1g

Configure & install OpenSSL

sudo ./config --prefix=/usr/local/ssl --openssldir=/usr/local/ssl shared zlib
sudo make
sudo make test
sudo make install

Configure shared libraries

Navigate to /etc/ld.so.conf.d and run the following

sudo echo "/usr/local/ssl/lib" >> /etc/ld.so.conf.d/openssl-1.1.1g.conf

ldconfig is used to create, update and remove symbolic links for the current shared libraries based on the lib directories present in the /etc/ld.so.conf

Reload with verbose

sudo ldconfig -v

Configure OpenSSL binary

Start with backing up the current OpenSSL

sudo mv /bin/openssl /bin/openssl.backup

Create script which will be executed on the system…

sudo vi /etc/profile.d/openssl.sh

… and set contents to

OPENSSL_PATH="/usr/local/ssl/bin"
export OPENSSL_PATH
PATH=$PATH:$OPENSSL_PATH
export PATH

Once done we need to make sure that the script is allowed to be executed

sudo chmod +x /etc/profile.d/openssl.sh

Next reload the profile to get the openSSL new binary with your PATH

source /etc/profile.d/openssl.sh

Verify openSSL version

 which openssl
 openssl version -a

If you reached this moment then we are ready to move on the next part πŸ™‚

Compiling Nginx with extra modules

Create user under which the process will be running

useradd -s/sbin/nologin -d/usr/local/nginx -M nginx

Navigate to folder with nginx sources created during download of our pre-reqs and run the config command

./configure 
--user=nginx 
--group=nginx 
--error-log-path=/var/log/nginx/error.log 
--http-log-path=/var/log/nginx/access.log 
--sbin-path=/usr/local/nginx/nginx 
--pid-path=/usr/local/nginx/nginx.pid 
--with-pcre=../pcre-8.44 
--with-zlib=../zlib-1.2.11 
--with-http_auth_request_module 
--with-http_geoip_module 
--with-http_gzip_static_module 
--with-http_gunzip_module 
--with-http_realip_module 
--with-http_secure_link_module 
--with-http_slice_module 
--with-http_ssl_module 
--with-http_v2_module

The above compiles Nginx with extra modules. For a comprehensive list with detailed information about each of the extra modules please refer to official Nginx documentation.

Once the above command finishes run

  make
  make install

When the above process finishes you should have nginx installed in /usr/local/nginx

Initial configuration of Nginx

In order to use Nginx we need to configure it. Right now our system knows nothing about running it.

Run vi /etc/systemd/system/nginx.service and set the content to

[Unit]
Description=The NGINX HTTP and reverse proxy server
After=syslog.target network-online.target remote-fs.target nss-lookup.target
Wants=network-online.target

[Service]
Type=forking
PIDFile=/usr/local/nginx/nginx.pid
ExecStartPre=/usr/local/nginx/nginx -t
ExecStart=/usr/local/nginx/nginx
ExecReload=/usr/local/nginx/nginx -s reload
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target

Now when you run systemctl status nginx you should see our service available.

Go ahead and run it! Type systemctl start nginx

At this moment you should have Nginx running with extra modules compiled!

1

Kubernetes kind with Traefik

If you have been working with kubernetes you already know that sometimes you just need a quick way to spin a cluster.

Well if it is so you must have came across kind . It allows you to spin k8s cluster locally.

kind is a tool for running local Kubernetes clusters using Docker container β€œnodes”.

With just one line you can get and spin up your cluster πŸ™‚


Once thats up and running I was interested to have some more fun with Traefik however on the “kind” documentation you could see only subset was mentioned as supported ones ….

So this felt like a good sunday challenge πŸ™‚ since I really had some plans to explore Traefik and its potential more.

I started off by finding the details of creating a new cluster with exposed mapped ports

cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 30080
    hostPort: 88
    protocol: TCP
  - containerPort: 30443
    hostPort: 444
    protocol: TCP
- role: worker
- role: worker
- role: worker
EOF

After the above it will create you 4 “nodes” from which one ( in my case control plane node ) will have the mapping and the specific node label ( the node label you can change to your needs )

Once your cluster is ready you can apply the bundle from the code below.It does not have host defined for ingress route or the middleware for dashboard but it is something you can easily add with CRDs

# Traefik bundle
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutes.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRoute
    plural: ingressroutes
    singular: ingressroute
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: middlewares.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Middleware
    plural: middlewares
    singular: middleware
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutetcps.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteTCP
    plural: ingressroutetcps
    singular: ingressroutetcp
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressrouteudps.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteUDP
    plural: ingressrouteudps
    singular: ingressrouteudp
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsoptions.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSOption
    plural: tlsoptions
    singular: tlsoption
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsstores.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSStore
    plural: tlsstores
    singular: tlsstore
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: traefikservices.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TraefikService
    plural: traefikservices
    singular: traefikservice
  scope: Namespaced
---
apiVersion: v1
kind: Namespace
metadata:
  name:  traefik
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - traefik.containo.us
    resources:
      - middlewares
      - ingressroutes
      - traefikservices
      - ingressroutetcps
      - ingressrouteudps
      - tlsoptions
      - tlsstores
    verbs:
      - get
      - list
      - watch
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name:  traefik
  namespace: traefik
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik
subjects:
  - kind: ServiceAccount
    name: traefik
    namespace: traefik
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: traefik
  namespace: traefik
spec:
  replicas: 1
  strategy:
    type: RollingUpdate
  selector:
    matchLabels:
      app.kubernetes.io/name: traefik
  template:
    metadata:
      labels:
        app.kubernetes.io/name: traefik
    spec:
      containers:
        - args:
            - --global.checknewversion
            - --global.sendanonymoususage
            - --entryPoints.traefik.address=:9000/tcp
            - --entryPoints.web.address=:9080/tcp
            - --entryPoints.websecure.address=:9443/tcp
            - --api.dashboard=true
            - --ping=true
            - --providers.kubernetescrd
            - --providers.kubernetesingress
            - --log.level=DEBUG
          image: traefik:2.2.11
          imagePullPolicy: IfNotPresent
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /ping
              port: 9000
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 2
          name: traefik
          ports:
          - containerPort: 9000
            name: traefik
            protocol: TCP
          - containerPort: 9080
            name: web
            protocol: TCP
          - containerPort: 9443
            name: websecure
            protocol: TCP
          readinessProbe:
            failureThreshold: 1
            httpGet:
              path: /ping
              port: 9000
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 2
          resources: {}
          securityContext:
            capabilities:
              drop:
              - ALL
            readOnlyRootFilesystem: true
            runAsGroup: 65532
            runAsNonRoot: true
            runAsUser: 65532
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      tolerations:
        - key: "node-role.kubernetes.io/master"
          effect: "NoSchedule"
          operator: "Exists"
      nodeSelector:
        ingress-ready: "true"
      restartPolicy: Always
      securityContext:
        fsGroup: 65532
      serviceAccountName: traefik
---
apiVersion: v1
kind: Service
metadata:
  name: traefik
  namespace: traefik
spec:
  type: NodePort
  selector:
    app.kubernetes.io/name: traefik
  ports:
    - name: web
      port: 9080
      nodePort: 30080
    - name: websecure
      port: 9443
      nodePort: 30443
---
apiVersion: v1
kind: Service
metadata:
  name: traefik-dashboard
  namespace: traefik
  labels:
    app.kubernetes.io/name: traefik
spec:
  type: ClusterIP
  ports:
  - port: 9000
    name: traefik
  selector:
     app.kubernetes.io/name: traefik
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-dashboard
spec:
  routes:
  - match: (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
    kind: Rule
    services:
    - name: [email protected]
      kind: TraefikService
    # middlewares:
    #   - name: auth
# ---
# apiVersion: traefik.containo.us/v1alpha1
# kind: Middleware
# metadata:
#   name: auth
# spec:
#   basicAuth:
#     secret: secretName # Kubernetes secret named "secretName"

Once the installation is completed you can access the dashboard. Of course that dashboard is exposed via IngressRoute so the fun can begin πŸ™‚

Hope this will enable you to quickly discover Traefik/Kubernetes/Kind πŸ™‚

Happy coding!