Configuration of Ambassador Edge Stack with EKS

Published in

Emumba

9 min readDec 7, 2023

There have been many blogs written about the importance of the role that a front proxy or API gateway can play in the delivery of software. It has played a vital role in the Continuous Delivery of modern applications. Almost any kind of software can be continuously delivered.

Introduction of Ambassador Edge Stack:

Ambassador Edge Stack is majorly useful in routing and securing traffic with a Kubernetes-native API Gateway built on Envoy proxy.

Envoy Proxy:

Envoy proxy is a high performance C++ distributed proxy designed for single services and applications, as well as a communication bus and universal data plane designed for large microservices “service mesh” architectures.

Features of Envoy proxy:

Out of process Architecture
HTTP/2 and GRPC Support
Advanced load balancing
APIS for configuration management

So, Ambassador Edge Stack is a specialized control plane for Envoy proxy. Ambassador edge stack translates configuration (in the form of Kubernetes custom resources) to Envoy configuration. All actual traffic is handled by high performance Envoy proxy.

Ambassador Edge Stack Architecture:

Features of Ambassador Edge Stack:

Some of the most highlighted features of Ambassador Edge Stack are:

Scalability: Manage high traffic volumes and distribute incoming requests across multiple backend services, ensuring smooth and reliable performance.
Security: Get robust security features, such as web application firewall (WAF), rate limiting, IP whitelisting, and SSL/TLS, to protect APIs from unauthorized access and malicious attacks.
Productivity & developer experience: Streamline developer productivity and enable self service options to leverage your teams’s existing Kubernetes workflows for configuration, deployment, and management of APIs.

Why Edge Stack API Gateway:

Ambassador Edge Stack provides various benefits and the following are the main reasons to use it:

Flexible Kubernetes management with ingress controller
Simplified traffic management to ensure high availability
Decentralized, declarative workflows
Edge policy management
Self-service control and flexibility
Professional world-class support

Distinctive Feature of Edge Stack:

One of the most distinctive features of Edge Stack is Decentralized and declarative workflows.

Some of the features of this are as follows:

Configure using Kubernetes Custom Resource Definitions (CRDs)
Operator-focused CRDs, including host, listener, and security configuration
Developer-focused CRDs, including mappings and rate limiting configuration
Comprehensive protocol support, including TCP, HTTP/1, HTTP/2, HTTP/3, and gRPC and gRPC-Web
Zero-downtime reconfiguration
Integrate with existing GitOps / Kubernetes workflows

Simplified Architecture of Ambassador Edge Stack:

Steps to setup Ambassador Edge Stack with EKS:

The following are the steps for deploying a fully functional Ambassador Edge Stack on EKS.

Step 1: Create Ambassador Cloud account & get token

Ambassador Edge Stack requires a license to function, so the first step is getting one to use while installing.

Log in to Ambassador Cloud with GitHub, GitLab or Google and select your team account.
Follow the prompts to name the cluster and click Generate Key.
Copy the token out and follow the next steps.
Once your cluster is connected to Ambassador Cloud, a community license is automatically applied.

Step 2: Installing the pre requisites for Edge Stack Installation:

The following are the prerequisites for Ambassador Edge Stack installation. Make sure to install them:

AWS CLI ( Documentation )
Kubectl Utility ( Documentation )
EKS Cluster in ready state.
Configure cluster using KUBECONFIG file ( For EKS, follow Documentation )

Step 3: Installing Ambassador Edge Stack on EKS

We can install Edge Stack from either Helm or Kubectl. The recommended method by Ambassador documentation is through Helm.

Installation using Kubernetes manifests can be consulted from the documentation.

Apply the following commands one by one one on your Kubernetes Cluster.

# Add the Repo:
helm repo add datawire https://app.getambassador.io
helm repo update
 
# Create Namespace and Install:
kubectl create namespace ambassador && \
kubectl apply -f https://app.getambassador.io/yaml/edge-stack/3.9.1/aes-crds.yaml
 
kubectl wait --timeout=90s --for=condition=available deployment emissary-apiext -n emissary-system
 
helm install edge-stack --namespace ambassador datawire/edge-stack \
  --set emissary-ingress.agent.cloudConnectToken=$CLOUD_CONNECT_TOKEN && \
kubectl -n ambassador wait --for condition=available --timeout=90s deploy -l product=aes

At this point, you have installed Edge Stack. Now let’s route some traffic to your services 🙂

Step 4: Confirming the Edge Stack Installation

Follow the simple commands to verify the Edge Stack installation on your Kubernetes cluster.

Edge Stack Services: Use kubectl get svc -n ambassador command to see the services of edge stack in the ambassador namespace.
Edge Stack Pods: Use kubectl get pods -n ambassador command to see pods associated with the edge stack service. Make sure that all pods are in running condition.

Step 5: Installing Listeners for Edge Stack:

Edge stack needs to listen on certain ports. For their configuration apply the following Kubernetes Yaml files. These can also be consulted from documentation.

Note: You can create the Kubernetes Yaml files by creating separate files with an extension of .yml or .yaml and pasting the content in it. Finally apply them using kubectl apply -f <file-name>

Apply the listener for HTTP & HTTPS traffic:

kubectl apply -f - <<EOF
---
apiVersion: getambassador.io/v3alpha1
kind: Listener
metadata:
  name: edge-stack-listener-8080
  namespace: ambassador
spec:
  port: 8080
  protocol: HTTP
  securityModel: XFP
  hostBinding:
    namespace:
      from: ALL
---
apiVersion: getambassador.io/v3alpha1
kind: Listener
metadata:
  name: edge-stack-listener-8443
  namespace: ambassador
spec:
  port: 8443
  protocol: HTTPS
  securityModel: XFP
  hostBinding:
    namespace:
      from: ALL
EOF

2. If you are using HTTP only protocol, apply the following Kubernetes Yaml file to cluster.

kubectl apply -f - <<EOF
---
apiVersion: getambassador.io/v3alpha1
kind: Listener
metadata:
  name: edge-stack-listener-8080
  namespace: ambassador
spec:
  hostBinding:
    namespace:
      from: ALL
  port: 8080
  protocol: HTTP
  securityModel: INSECURE
EOF

Note: These files just need to be copied and pasted in the terminal. They’ll be automatically applied because they contains EOF arguments.

Step 6: Installing & Configuring AWS ALB Controller

If you are working with EKS, you need to install the AWS ALB Controller in the EKS namespace kube-system for our ingress ALB to work fine. This can also be consulted from the documentation here.

For this, we can follow the following steps:

Create an IAM Policy:

curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.5.4/docs/install/iam_policy.json

2. Create an IAM policy using policy downloaded in the previous step:

aws iam create-policy \
    --policy-name AWSLoadBalancerControllerIAMPolicy \
    --policy-document file://iam_policy.json

3. Retrieve your cluster’s OIDC provider ID and store it in a variable. Replace my-cluster with actual name of your EKS cluster.

oidc_id=$(aws eks describe-cluster --name my-cluster --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5)

4. Determine whether an IAM OIDC provider with your cluster’s ID is already in your account.

aws iam list-open-id-connect-providers | grep $oidc_id | cut -d "/" -f4

If output is returned, then you already have an IAM OIDC provider for your cluster. In case no output is returned, then you must create an IAM OIDC provider for your cluster. For more information, you can consult here.

5. Copy the following contents to your device. Replace 111122223333 with your account ID. Replace region-code with the AWS Region that your cluster is in. Replace EXAMPLED539D4633E53DE1B71EXAMPLE with the output returned in the previous step. After replacing the text, run the modified command to create the load-balancer-role-trust-policy.json file.

cat >load-balancer-role-trust-policy.json <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::111122223333:oidc-provider/oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:aud": "sts.amazonaws.com",
                    "oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub": "system:serviceaccount:kube-system:aws-load-balancer-controller"
                }
            }
        }
    ]
}
EOF

6. Create the IAM role.

aws iam create-role \
  --role-name AmazonEKSLoadBalancerControllerRole \
  --assume-role-policy-document file://"load-balancer-role-trust-policy.json"

7. Attach the required EKS managed IAM policy to the IAM role. Replace 111122223333 with your account ID.

aws iam attach-role-policy \
  --policy-arn arn:aws:iam::111122223333:policy/AWSLoadBalancerControllerIAMPolicy \
  --role-name AmazonEKSLoadBalancerControllerRole

8. Copy the following contents to your device. Replace 111122223333 with your account ID. After replacing the text, run the modified command to create aws-load-balancer-controller-service-account.yaml file

cat >aws-load-balancer-controller-service-account.yaml <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/name: aws-load-balancer-controller
  name: aws-load-balancer-controller
  namespace: kube-system
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::111122223333:role/AmazonEKSLoadBalancerControllerRole
EOF

9. Create the Kubernetes service account on your cluster. The Kubernetes service account named aws-load-balancer-controller is annotated with the IAM role that you created named AmazonEKSLoadBalancerControllerRole

kubectl apply -f aws-load-balancer-controller-service-account.yaml

10. Install the AWS Load Balancer Controller using Helm V3 or later or by applying a Kubernetes manifest as in the documentation.

helm repo add eks https://aws.github.io/eks-charts

11. Update your local repo to make sure that you have the most recent charts.

helm repo update eks

12. Replace, my-cluster with the name of your cluster. In the following command, aws-load-balancer-controller is the Kubernetes service account that you created in the previous step.

helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=my-cluster \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller

13. Verify that the controller is installed correctly.

kubectl get deployment -n kube-system aws-load-balancer-controller

An example output is as follows:

NAME                           READY   UP-TO-DATE   AVAILABLE       AGE
aws-load-balancer-controller   2/2      2             2             84s

Step 7: Installing the Ambassador ALB using ingress

We need to create the ALB so that we can access our services from outside the cluster using prefixes.

The pre-requisite for this step is to have an AWS Load Balancer Controller installed and configured in the EKS cluster which can be done by following Step 6 or from these links:

2. Once the AWS Load Balancer Controller is installed, we can write the ingress file for creating an Application Load Balancer (ALB) using ingress by following the sample code:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ambassador-ingress
  namespace: ambassador
  annotations:
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/scheme: internet-facing  # Use "internal" for internal ALB
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
          - path: /*
            pathType: ImplementationSpecific
            backend:
              service:
                name: edge-stack  
                port:
                  number: 80

The file can be applied by using kubectl apply -f <file-name.yml>
The DNS for the ALB created can be found from the AWS Management Console in Load Balancers section of EC2 or by using command:

kubectl get ingress -n ambassador

Step 8: Creating the HOST for Ambassador Edge Stack

The Edge Stack requires a host that can be linked to the mapping to ensure proper communication between them.

More documentation of HOSTs can be found here.

For the creation of a basic host, you can follow the following steps:

Create the Host based on wildcard. You need to create its Kubernetes manifest with extension of .yml or .yaml.

---
apiVersion: getambassador.io/v3alpha1
kind: Host
metadata:
  labels:
    app.kubernetes.io/instance: edge-stack
  name: ambassador
  namespace: ambassador
spec:
  acmeProvider:
    authority: none
  ambassador_id:
  - default
  hostname: '*'

The file can be applied using the command kubectl apply -f <file-name.yml>

2. Create the Host based on ALB and mapping selector. This is useful for allowing only the DNS of Load Balancer to redirect the requests to the Host and then to the mappings.

---
apiVersion: getambassador.io/v3alpha1
kind: Host
metadata:
  labels:
    app.kubernetes.io/instance: edge-stack
  name: ambassador-host
  namespace: ambassador
spec:
  acmeProvider:
    authority: none
  ambassador_id:
  - default
  hostname: 'Your Ambassador ALB  DNS'
  requestPolicy:
    insecure:
      action: Route
  mappingSelector:
    matchLabels:
      host: ambd

The file can be applied using the command kubectl apply -f <file-name.yml>

Step 9: Creating the mappings for the services

Suppose, we have a simple service running on EKS with the name of main-service.

More detailed information on how to set up services on Kubernetes can be consulted from here.

The mapping for this service (main-service) can be created as:

---
apiVersion: getambassador.io/v3alpha1
kind: Mapping
metadata:
  name: main-service-mapping
  namespace: ambassador
  labels:
    host: ambd
spec:
  ambassador_id:
  - --apiVersion-v3alpha1-only--default
  connect_timeout_ms: 18000
  host: "Your Ambassador ALB DNS"
  prefix: /
  service: main-service.test:80
  timeout_ms: 18000
  retry_policy:
    num_retries: 10
    retry_on: 5xx

Note: In the above mapping, service: main-service.test:80 is decoded as:

Service Name: main-service
Namespace: test
Port on which it is listening: 80

More detailed information on Mapping can be consulted from official documentation of the Ambassador Edge Stack here.

Step 10: Test the deployments using ALB

The deployments can be tested using ALB Endpoint and prefixes.

The ALB created in Step 7 can be accessed in two ways.

It can be seen in the Load Balancers section of EC2 service of AWS console.
It can also be seen using the command:

kubectl get ingress -n ambassador

2. Copy the DNS name from either of the above ways and access your services with the prefix assigned in the mappings.

For Example, for our main-service, the prefix associated with mapping is / so it can be accessed using http://DNS_OF_ALB/

Conclusion:

Thats it, in this way, you can configure a fully functional ambassador edge stack service to run on your EKS cluster with proper configuration of all associated dependencies.🙂