Comprehensive Guide to Secure HTTPBin with OAuth2 Proxy and Keycloak using OpenID Connect

Step-by-step integration and testing of secure HTTPBin using OAuth2 Proxy and Keycloak OIDC

Highlights

Seamless Integration: Configure Keycloak as the OIDC provider and OAuth2 Proxy as the reverse proxy to secure your HTTPBin application.
Step-by-Step Setup: Detailed instructions covering Keycloak client creation, OAuth2 Proxy configuration, and HTTPBin deployment to ensure a robust security setup.
Testing and Troubleshooting: Methods to test the security layer through authentication redirects, access token validation, and header verification.

Introduction

Securing an application, even one like HTTPBin which serves as a testing endpoint, is essential particularly when simulating production environments. In this guide, we will walk you through configuring a secure environment where HTTPBin is protected by an OAuth2 Proxy integrated with Keycloak using OpenID Connect (OIDC). This configuration offloads the authentication and authorization process to dedicated providers ensuring a seamless, secure, and flexible authentication workflow.

The process leverages Keycloak as the identity provider. Keycloak supports OpenID Connect natively and can issue, manage, and validate tokens while OAuth2 Proxy acts as an authentication gateway intercepting unauthenticated requests. Once authenticated, the proxy attaches necessary authentication headers before forwarding requests to HTTPBin. The following sections will cover prerequisites, detailed configuration steps, testing methods, and troubleshooting techniques to ensure a smooth and secure setup.

Prerequisites and Environment Setup

Before diving into the detailed configurations, ensure you have the following components and tools available:

A working installation of Keycloak (standalone server, Docker container, or installed in a Kubernetes cluster).
Deployment method for OAuth2 Proxy (Docker/Kubernetes/standalone)
An instance of HTTPBin. This can be the public HTTPBin or a self-hosted instance using Docker, Kubernetes, or similar.
Basic infrastructure preparation such as DNS resolution, SSL certificates for HTTPS (especially in production) and network connectivity between services.
Command line tools such as curl, or Postman for testing API endpoints.

Creating the Required Namespaces and Deploying Components

If you are using Kubernetes, it’s beneficial to isolate your components into different namespaces. This separation can enhance management and scalability in production environments.


  # Create namespaces for segregation
  kubectl create ns keycloak
  kubectl create ns oauth-proxy
  kubectl create ns httpbin

In non-Kubernetes environments, ensure that your network environment or virtual machines are arranged to logically separate these services. The following sections provide detailed configurations for both Keycloak and OAuth2 Proxy.

Setting Up Keycloak

Keycloak Installation

The first step is installing Keycloak. Depending on your environment, you can either run Keycloak as a Docker container, use a pre-built server binary, or deploy it in Kubernetes using Helm.

For Kubernetes installations, you can use a Helm chart. Below is an example command using the Bitnami Helm chart:


  # Add the Bitnami repository and update the repo list
  helm repo add bitnami https://charts.bitnami.com/bitnami
  helm repo update

  # Install Keycloak in the keycloak namespace
  helm install keycloak bitnami/keycloak --namespace keycloak

Configuring the Keycloak Realm and Client

Once Keycloak is installed, you need to perform the following configurations within Keycloak:

Create a Realm: A realm in Keycloak is a space where you manage objects such as users, applications (referred to as clients), and roles. Create a realm suitable for your testing environment.
Create a Client: Under your new realm, add a client application. The client represents the OAuth2 Proxy. Set it up as follows:
- Client ID: oauth2-proxy (or a relevant identifier)
- Client Protocol: OpenID Connect
- Access Type: Confidential
- Valid Redirect URIs: This should include the OAuth2 Proxy callback URL. For example, if your OAuth2 Proxy is hosted at http://oauth2-proxy.example.com:4180, then append /oauth2/callback (i.e., http://oauth2-proxy.example.com:4180/oauth2/callback).
Configure Client Scopes and Mappers:
Optionally, add a group-based mapper if you want to implement granular access control. This mapper ensures that group membership is carried in the token which can be used by OAuth2 Proxy for group-based authorization.

The Keycloak admin console provides step-by-step wizard options to configure these settings. Ensure that you note the generated client secret as it will be used during the OAuth2 Proxy configuration.

Configuring OAuth2 Proxy

Deploying OAuth2 Proxy

OAuth2 Proxy acts as a reverse proxy, intercepting requests for secure services and managing the authentication flow. You can deploy it using Docker, Kubernetes (via Helm), or as a stand-alone binary.

For a Helm-based Kubernetes deployment, you can install OAuth2 Proxy with the following command:


  helm repo add oauth2-proxy https://oauth2-proxy.github.io/manifests
  helm repo update

  helm install oauth2-proxy oauth2-proxy/oauth2-proxy --namespace oauth-proxy

OAuth2 Proxy Configuration

OAuth2 Proxy must be configured to use Keycloak as its OIDC provider. A typical configuration file in YAML might look like the following:


  # oauth2-proxy-values.yaml
  config:
    # Basic OAuth2 Proxy settings
    clientID: "oauth2-proxy"
    clientSecret: "YOUR_CLIENT_SECRET"
    cookieSecret: "a_random_generated_secret"
    # The file-based configuration for additional settings
    configFile: |
      provider = "oidc"
      oidc_issuer_url = "http://keycloak.example.com:8080/auth/realms/your_realm"
      redirect_url = "http://oauth2-proxy.example.com:4180/oauth2/callback"
      upstreams = "http://httpbin-service"
      # Optional: Pass additional options like email_domains if needed
      email_domains = ["*"]

Replace YOUR_CLIENT_SECRET with the client secret generated in Keycloak. Adjust the URL endpoints based on your operational environment. It is important that OAuth2 Proxy’s redirect_url matches the one configured in Keycloak.

Key Parameters Explained

Parameter	Description
clientID	The identifier for the OAuth2 Proxy client as configured in Keycloak.
clientSecret	The secret associated with the client in Keycloak.
cookieSecret	A random string used to secure cookies issued by OAuth2 Proxy.
oidc_issuer_url	URL of the OIDC provider, typically your Keycloak realm endpoint.
redirect_url	The endpoint on OAuth2 Proxy where Keycloak will redirect after authentication.
upstreams	The backend HTTP service (HTTPBin) that will be protected.

Deploying and Securing HTTPBin

HTTPBin Deployment

HTTPBin is a simple HTTP request and response service that is excellent for testing. Deployment can be done using Docker, Kubernetes, or directly on a server.

For a Kubernetes deployment, you can use the following commands:


  # Deploy HTTPBin in the httpbin namespace
  kubectl create deployment httpbin --image=kennethreitz/httpbin -n httpbin

  # Expose the HTTPBin service on port 80
  kubectl expose deployment httpbin --port 80 -n httpbin

Integrating HTTPBin with OAuth2 Proxy

Once HTTPBin is deployed, ensuring that it is protected is as simple as routing traffic through the OAuth2 Proxy. Any request to HTTPBin must first go to OAuth2 Proxy, which will redirect unauthenticated requests to Keycloak.

In your OAuth2 Proxy configuration, the upstreams parameter should point directly to the HTTPBin service URL. For example, if your HTTPBin service is accessible internally via a service name like httpbin in Kubernetes, then upstreams = "http://httpbin" is valid.

Testing the Secure Setup

Authentication Flow Testing

A proper test of the secure setup involves verifying that:

Unauthenticated access attempts are redirected to the Keycloak login page.
Successful login in Keycloak redirects back to OAuth2 Proxy, attaching valid authentication headers.
The HTTPBin service is accessible only once the user is authenticated.

To test the setup:

Open up your web browser and navigate to http://oauth2-proxy.example.com:4180/. An unauthenticated request should automatically redirect you to the Keycloak login page.
Enter valid user credentials. Once authenticated, your browser will be redirected back to the OAuth2 Proxy and then to HTTPBin.
Verify the authentication headers. HTTPBin’s /headers endpoint can show the headers passed along after authentication. Access http://oauth2-proxy.example.com:4180/headers and confirm that headers (such as X-Forwarded-User or other OIDC-related headers) are present.

Additionally, testing can be performed via curl commands:


  # Obtain an access token (if needed) from Keycloak using its token endpoint
  curl -X POST "http://keycloak.example.com:8080/auth/realms/your_realm/protocol/openid-connect/token" \
       -d "client_id=oauth2-proxy" \
       -d "client_secret=YOUR_CLIENT_SECRET" \
       -d "username=YOUR_USERNAME" \
       -d "password=YOUR_PASSWORD" \
       -d "grant_type=password" | jq -r .access_token

  # Use the access token to call HTTPBin via OAuth2 Proxy
  curl -i http://oauth2-proxy.example.com:4180/get \
       -H "Authorization: Bearer YOUR_ACCESS_TOKEN"

Testing Group Based Access (Optional)

If group-based authorization is enabled in Keycloak, ensure that the token carries group membership. OAuth2 Proxy can be configured to restrict access based on these groups. For instance, by using the --allowed-group option, you can restrict which users can access certain endpoints, ensuring enhanced security.

Troubleshooting and Advanced Considerations

Common Issues and Resolutions

While deploying the secure HTTPBin setup, you may encounter several issues. Here are the most common challenges and their solutions:

Mismatched Redirect URLs: Ensure that the redirect URL configured in Keycloak exactly matches the OAuth2 Proxy callback URL.
Invalid Client Credentials: Double-check the client secret and client ID in both Keycloak and OAuth2 Proxy configurations.
Missing Authentication Headers: Verify that OAuth2 Proxy is configured to forward the necessary headers to HTTPBin. Inspect these headers using HTTPBin’s /headers endpoint.
Network Issues: If using Kubernetes, ensure that services in separate namespaces can communicate. Forward ports properly and ensure DNS resolution is functional.
SSL/TLS Problems: In production, ensure SSL certificates are properly configured for both Keycloak and OAuth2 Proxy if HTTPS is enforced.

Security Enhancements

To further improve security:

Implement HTTPS on all endpoints. Use valid SSL certificates to avoid man-in-the-middle attacks.
Monitor logs of both Keycloak and OAuth2 Proxy for any suspicious activity.
Regularly update and patch your Keycloak server, OAuth2 Proxy, and any related dependencies.
Configure rate limiting and WAF (Web Application Firewall) rules at the reverse proxy level for additional protection.

Additional Configuration Examples and Best Practices

Using Docker Compose for a Local Setup

For local testing, Docker Compose can simplify the deployment of Keycloak, OAuth2 Proxy, and HTTPBin. Here is an example of a Docker Compose file:


  version: '3.8'
  services:
    keycloak:
      image: quay.io/keycloak/keycloak:latest
      environment:
        KEYCLOAK_USER: admin
        KEYCLOAK_PASSWORD: admin
      ports:
        - "8080:8080"
      command: start-dev

    oauth2-proxy:
      image: quay.io/oauth2-proxy/oauth2-proxy:latest
      ports:
        - "4180:4180"
      environment:
        OAUTH2_PROXY_PROVIDER: oidc
        OAUTH2_PROXY_OIDC_ISSUER_URL: http://keycloak:8080/realms/your_realm
        OAUTH2_PROXY_CLIENT_ID: oauth2-proxy
        OAUTH2_PROXY_CLIENT_SECRET: YOUR_CLIENT_SECRET
        OAUTH2_PROXY_COOKIE_SECRET: a_random_generated_secret
        OAUTH2_PROXY_REDIRECT_URL: http://localhost:4180/oauth2/callback
        OAUTH2_PROXY_UPSTREAMS: http://httpbin:80

    httpbin:
      image: kennethreitz/httpbin
      ports:
        - "80:80"

Execute docker-compose up and then access the service at http://localhost:4180 to test the entire authentication flow.

Additional Notes for Production Environments

When moving from a development/testing environment to production:

Secure communications using HTTPS for all endpoints. Configure TLS termination at your load balancers or reverse proxies.
Define strict firewall rules to limit access to Keycloak and OAuth2 Proxy.
Employ logging, monitoring, and auditing solutions to keep track of authentication flows and potential anomalies.
Regularly review and adhere to security best practices for both Keycloak and OAuth2 Proxy as recommended by their official documentation.

Conclusion and Final Thoughts

This guide provided an extensive walkthrough for testing and securing HTTPBin with OAuth2 Proxy and Keycloak using OpenID Connect. The process involves setting up and configuring Keycloak as an identity provider, deploying OAuth2 Proxy as a reverse proxy to enforce authentication, and finally deploying HTTPBin as the protected backend service. It also walked through testing the setup by using both browser-based flows and command-line tools to ensure the security layer functions as intended.

By following these detailed steps, you not only secure your HTTPBin instance but also gain insights into a production-grade architecture that can be extended to other applications. This setup allows you to offload authentication, simplify access management, and leverage a robust OIDC implementation provided by Keycloak, ensuring that your services are adequately protected against unauthorized access.

Adhering to these configurations and best practices enhances overall security, scalability, and maintainability of your secure environments. As security requirements evolve, continuous monitoring and adjustments to configurations are recommended to combat emerging threats effectively.