Container-Native Metering

Revenium's container-native metering agent offers real-time, zero-latency monitoring and metering of API and microservice traffic across without requiring API gateways, sidecars or code modification

With Revenium's container agent, API and microservice traffic can be monitored and metered on Linux bare metal, virtual machines and containers. This provides real-time, zero-latency visibility into API traffic across an enterprise without the need for API Gateways, sidecars, code modification or other invasive approaches.

For testing purposes, our container-based instructions are simple to implement, though for production use cases, we recommend a native installation on the host server to allow for all container traffic to be metered without individual modification. There are also security reasons for this recommendation which are explained in the Docker Installation section below.

Each of the installation options is outlined below.

Looking for a quick implementation & test? Revenium provides a Docker Compose stack that lets you experience Revenium's native metering solution in a self-contained environment that will send metering events to your Revenium account. The stack will create one container to send API traffic, one to receive traffic and a synthetic load generator. You can download the repository using the instructions below.

Installation in Linux

Installation Instructions (Debian)

Replace HTTP_PORT with the port your APIs are bound to and REVENIUM_API_KEY with your Revenium API Key.

curl -sSL https://raw.githubusercontent.com/hypercurrentio/isotope-files/main/install_and_run  | sudo bash -s -- HTTP_PORT REVENIUM_API_KEY

Installation Instructions (APT)

Its also possible to install the agent manually using apt:

Step 1: Update and install necessary packages

apt-get update && apt-get install -y curl lsb-release

Step 2: Download the keyring for your package repository

curl -Lo /usr/share/keyrings/rm-dev-archive-keyring.gpg https://pkg.dev.hcapp.io/deb/rm-dev-archive-keyring.gpg

Step 3: Add the Revenium package repository to the sources list

printf "deb [signed-by=%s] %s/%s %s main\n" \
    "/usr/share/keyrings/rm-dev-archive-keyring.gpg" \
    "https://pkg.dev.hcapp.io/deb" \
    "$(lsb_release -is | tr A-Z a-z)" \
    "$(lsb_release -cs)" \
    > "/etc/apt/sources.list.d/rm-dev.list"

Step 4: Update and install the agent and its eBPF shim

apt-get update && apt-get install -y isotope isotope-ebpf-shim

RPM installation for RedHat derivatives is coming soon! Let us know if you need it right now.

Starting the Agent in Linux

The agent can be run from /usr/bin/isotope and accepts the following command line parameters.

Command Line Parameters:

 --api-key string
    	The Revenium Platform API key
  --correlation-header string
    	An optional HTTP header used to correlate API responses to requests.  If unset responses will be correlated to requests based on their arrival sequence
  --dequeuing-interval int
    	The interval in seconds to dequeue metering events (default is 5 seconds) (default 5)
  --interfaces string
    	Comma-separated list of network interfaces to listen for API traffic on
  --max-concurrency int
    	The amount of API events to process in parallel (default 10)
  --metering-header string
    	The HTTP header that identifies the API Subscriber (default "clientId")
        Note that this is a critical header to define to maximize your analytics value
  --platform-api-url string
    	The Revenium Platform API URL (default "https://api.revenium.io")
  --port-mappings string
    	Comma-separated list of mapped -> destination ports (ie: 9091:8080,9443:8443)
  --ports string
    	Comma-separated list of ports to listen for API traffic on (default "80")
  --pprof
    	capture runtime profiling data (debug use only)
  --statistics-interval int
    	The interval in seconds to log runtime statistics (if 0 statistic logging is disabled)

The agent supports binding to the Docker interface (ie, "docker0") to meter intra-container API traffic

The following shows the agent listening on ports 8080 and 8443 on the "eth0" and "docker0" interfaces and emitting statistics to stdout every 5 seconds. Additionally, it defines a port mapping that will decode requests on port 9091 to port 8080 so that external traffic arriving on port 9091 mapped to port 8080 can be captured. Lastly, it defines the header (clientName) that will send the client identifier to Revenium so that you can easily identify API traffic by Subscriber.

/usr/bin/isotope --ports 8080,8443 --interfaces eth0,docker0 --port-mappings 9091:8080 --api-key REVENIUM_API_KEY --statistics-interval 5 --metering-header clientName

Installing Native Metering in Docker

An entrypoint script can be used to integrate the Revenium agent into an API's Docker image. Download the entrypoint script template into the directory with your Dockerfile using the command below.

curl https://raw.githubusercontent.com/hypercurrentio/revenium-isotope-sandbox/main/entrypoint > entrypoint

Next modify the entrypoint script with the HTTP port your API is listening on and your Revenium API Key and save the entrypoint file in the directory with your Dockerfile and docker-compose.yml file.

#!/bin/sh
curl -sSL https://raw.githubusercontent.com/hypercurrentio/isotope-files/main/install_and_run \
| sh -s -- HTTP_PORT REVENIUM_API_KEY
exec "$@"

Command line parameters can also be appended to the entrypoint script. In the example immediately below we are configuring the agent to listen for HTTP API traffic on ports 8080 and 9090 and also specifying the correlation header we'll use to map API requests to responses. Lastly, it defines the header (clientName) that will send the client identifier to Revenium so that you can easily identify API traffic by Subscriber.

#!/bin/sh
curl -sSL https://raw.githubusercontent.com/hypercurrentio/isotope-files/main/install_and_run \
| sh -s -- HTTP_PORT REVENIUM_API_KEY --ports 8080,9090 --correlation-header x-revenium-correlation-id -metering-header clientName
exec "$@" 

In the absence of a correlation header, the agent correlates HTTP requests with responses using TCP arrival sequence and source/destination ports.

Next, add the entrypoint to your Dockerfile. An example Dockerfile is shown below with lines 14 & 15 added to show what you will need to insert this into your own environment.

FROM python:3.8-slim AS rm-isotope-sandbox

ARG DEBIAN_FRONTEND=noninteractive
ARG APT_LISTCHANGES_FRONTEND=none
RUN apt-get -qq update\
 && apt-get -qq install\
 curl\
 lsb-release\
 && apt-get -qq clean

COPY --link app/ /usr/src/app/

# COPY AND INSTALL THE ENTRYPOINT FILE DOWNLOADED ABOVE
COPY --link --chmod=755 ./entrypoint /entrypoint
ENTRYPOINT ["/entrypoint"]

WORKDIR /usr/src/app
RUN pip install --no-cache-dir -r requirements.txt

EXPOSE 8080
ENV NAME EchoAPI

# Run app.py when the container launches
CMD ["python", "./app.py"]

Once you have inserted the lines above into your Dockerfile, proceed to build and run the docker images as you normally would with commands like docker build and docker run/ docker-compose up.

Once launched, the agent will be running in the background and logging to /var/log/isotope.log

Docker containers running eBPF programs usually require running the program in "privileged" mode and mounting the host's BPF filesystem (/sys/fs/bpf). Production deployments should run the agent on the container host (using the instructions above) in order to monitor API traffic across all containers without requiring privileged access.

Docker Compose Stack for Metering Testing

If you prefer to test in a sample docker environment before implementing this agent into your own application, you can easily do so using our metering sandbox repository on GitHub. The stack will create one container to send API traffic, one to receive traffic and a synthetic load generator, all of which are connected to your own Revenium account (linked by providing your API key in the configuration).

Testing Metering

To observe transactions being recorded in Revenium navigate to Transactions -> Analytics Transaction Logs.

You will also see analytics data from your transactions presented under the "API Analytics" section of the left navigation menu.