Extending Envoy Proxy with Golang WebAssembly

By Effi Bar-She’an

Envoy is a L7 proxy and communication bus designed for large modern service-oriented architectures.

Envoy can be used to monitor and control HTTP connections. One way to do this is using the Lua scripting language, for example to intercept requests and responses. Another option, is using a Web Assembly (WASM) plugin.

As Golang developers, we can develop our WASM plugin using Go SDK.

Let’s write and run an Envoy proxy with a WASM extension written in Go :-)

First, you need to install Envoy:

$ brew update
$ brew install envoy

Note, that alternatively you can work with Docker, but currently on MacOS there is an issue that makes it harder.

WASM extensions can’t be developed with regular Go, instead you use TinyGo:

$ brew tap tinygo-org/tools
$ brew install tinygo

Now, let’s write our main.go to log request headers:

package main

import (
   "github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm"
   "github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm/types"
)

type helloHttpContext struct {
   proxywasm.DefaultHttpContext
}

func main() {

   proxywasm.SetNewHttpContext(newHttpContext)
}

func newHttpContext(uint32, uint32) proxywasm.HttpContext {

   return &helloHttpContext{}
}

func (ctx *helloHttpContext) OnHttpRequestHeaders(numHeaders int, _ bool) types.Action {

   if numHeaders > 0 {
      headers, err := proxywasm.GetHttpRequestHeaders()
      if err != nil {
         proxywasm.LogErrorf("failed to get request headers with '%v'", err)
         return types.ActionContinue
      }
      proxywasm.LogInfof("request headers: '%+v'", headers)
   }

   return types.ActionContinue
}

helloHttpContext embeds proxywasm.DefaultHttpContext so that we do not need to implement all the methods of HttpContext.

The only method that we override is OnHttpRequestHeaders which will log the request headers on all requests that contain headers.

In order to get the request headers we use proxywasm.GetHttpRequestHeaders. Note, that we can’t retrieve the request body as part of OnHttpRequestHeaders method. To do that we need to override OnHttpRequestBody.

Let’s build our WASM with TinyGo:

$ tinygo build -o ./hello.wasm -scheduler=none -target=wasi ./main.go

This should create the hello.wasm file in the current directory.

Before running envoy we need to to create a config file envoy.yaml. This configuration loads envoy with a WASM filter and then listens to port 8085 as a reverse-proxy to backend service — hello that listens on localhost:8080:

static_resources:
  listeners:
    - name: listener_0
      address:
        socket_address: { address: 127.0.0.1, port_value: 8085 }
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_http
                codec_type: AUTO
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: local_service
                      domains: ["*"]
                      routes:
                        - match: { prefix: "/" }
                          route: { cluster: svc_hello }
                http_filters:
                  - name: envoy.filters.http.wasm
                    typed_config:
                      "@type": type.googleapis.com/udpa.type.v1.TypedStruct
                      type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
                      value:
                        config:
                          name: "my_plugin"
                          root_id: "my_root_id"
                          vm_config:
                            vm_id: "my_vm_id"
                            runtime: "envoy.wasm.runtime.v8"
                            code:
                              local:
                                filename: "hello.wasm"
                            allow_precompiled: true
                  - name: envoy.filters.http.router
                    typed_config: {}

  clusters:
    - name: svc_hello
      connect_timeout: 0.25s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: svc_hello
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 127.0.0.1
                      port_value: 8080

admin:
  access_log_path: "/dev/null"
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 8001

Next, here’s our backend hello service which listens on port 8080 and responds to /hello which our Envoy proxy will redirect all the traffic to:

package main

import (
   "net/http"

   log "github.com/sirupsen/logrus"
)

func main() {

   m := http.NewServeMux()
   m.HandleFunc("/hello", func(w http.ResponseWriter, r *http.Request) {
      if _, err := w.Write([]byte("hola mundo :)")); err != nil {
         log.Errorf("failed to stream response with '%v'", err)
      }
   })
   const addr = ":8080"
   server := &http.Server{
      Addr:    addr,
      Handler: m,
   }
   log.Infof("listening on '%s'", addr)
   log.Fatal(server.ListenAndServe())
}

After running the hello service, let’s run the Envoy proxy with our config file:

$ envoy -c envoy.yaml -l debug

Note, we’re running it with debug level debug, the default is info.

Calling our service:

$ curl localhost:8085/hello

should result with the follow Envoy’s logs:

wasm log: response headers: [[:status 200] [date Sun, 24 Jan 2021 00:42:00 GMT] [content-length 13] [content-type text/plain; charset=utf-8] [x-envoy-upstream-service-time 0]]

See full code in here.

References:

• Envoy
• TinyGo
• Go SDK for WebAssembly-based Envoy extensions
• Redefining extensibility in proxies - introducing WebAssembly to Envoy and Istio