NoETL Gateway

The NoETL Gateway is a Rust-based API gateway that provides authentication, authorization, GraphQL compatibility, and authenticated REST proxy access to the NoETL platform.

Source Code

For development documentation, local setup, and code details, see the Gateway Crate README.

Architecture

┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Browser   │────▶│  Cloudflare │────▶│   Gateway   │────▶│   NoETL     │
│             │     │   (Proxy)   │     │ (GKE/K8s)   │     │   Server    │
└─────────────┘     └─────────────┘     └─────────────┘     └─────────────┘
       │                                    │    │                │
       │                                    ▼    │                ▼
       │                           ┌────────────┐│         ┌─────────────┐
       │                           │  NATS K/V  ││         │  PostgreSQL │
       │                           │ (sessions) ││         │   (auth)    │
       └───────────────────────────┴────────────┘│         └─────────────┘
              (Authentication)                   │
                                        ┌────────┘
                                        ▼
                               ┌─────────────┐
                               │    Auth0    │
                               │  (Identity) │
                               └─────────────┘

Session Caching with NATS K/V

The Gateway uses NATS K/V as a fast session cache to avoid calling NoETL playbooks for every authenticated request:

Gateway Request → Check NATS K/V → Cache Hit? → Use cached session (sub-ms)
                                 → Cache Miss? → Call playbook → Refresh cache

Benefits:

• Sub-millisecond session lookups from NATS K/V
• Reduced load on NoETL server and PostgreSQL
• PostgreSQL remains source of truth for session data
• Automatic cache refresh via playbooks

Key Features

• Auth0 Integration: OAuth2/OIDC authentication via Auth0 Universal Login
• Session Caching: Fast session lookups via NATS K/V cache
• Session Management: Session tokens managed via NoETL playbooks (PostgreSQL source of truth)
• GraphQL Compatibility: Authenticated executePlaybook and proxy helpers for clients that prefer GraphQL
• REST Proxy: Canonical /noetl/* forwarding to NoETL server /api/*
• CORS Support: Configurable cross-origin resource sharing
• Stateless Design: No direct database connections

Agent and MCP Execution

Gateway does not call MCP servers directly. It authenticates the client and forwards canonical execution requests to NoETL:

POST /noetl/execute
{
  "path": "automation/agents/kubernetes/runtime",
  "workload": {
    "method": "tools/call",
    "tool": "pods_list_in_namespace",
    "arguments": { "namespace": "noetl" }
  },
  "resource_kind": "agent"
}

The NoETL server dispatches the playbook, the worker executes kind: mcp, and the resulting activity is tracked in NoETL execution state. This keeps GUI terminal commands, external API calls, and scheduled jobs on the same agent-as-playbook audit path.

Gateway's typed GraphQL executePlaybook mutation also accepts resourceKind; use agent for catalog entries registered as agent playbooks.

API Endpoints

Public Endpoints

Endpoint	Method	Description
/health	GET	Health check
/api/auth/login	POST	Auth0 token login
/api/auth/validate	POST	Validate session
/api/auth/check-access	POST	Check playbook permissions
/api/runtime/contract	GET	Gateway route and execution contract

Protected Endpoints (Require Authentication)

Endpoint	Method	Description
/graphql	POST	Execute playbooks via GraphQL
/graphql	GET	GraphiQL playground
/noetl/{path}	GET/POST/PUT/DELETE/PATCH	Proxy to NoETL server API

Real-time Callbacks (SSE)

Endpoint	Method	Description
/events	GET	SSE connection for real-time playbook results
/api/internal/callback/async	POST	Worker callback for async results
/api/internal/progress	POST	Worker progress updates

Documentation

Guide	Description
Deployment Guide	Building, deploying to GKE, static IP setup
Helm Reference	Complete Helm chart configuration
Auth0 Setup	Auth0 application and integration
Auth Integration	Auth playbooks and session management
Async Callbacks	SSE/WebSocket real-time playbook results
Cloudflare Setup	DNS, SSL, caching configuration
API Usage Guide	How to authenticate and call playbooks

Quick Start

Deploy to GKE

noetl run automation/iap/gcp/deploy_gke_stack.yaml \
  --set project_id=YOUR_PROJECT_ID \
  --set deploy_gateway=true \
  --set create_cluster=false \
  --set deploy_noetl=false

Local Development

# Run gateway locally
cd crates/gateway
cargo run

# Environment variables
export ROUTER_PORT=8090
export NOETL_BASE_URL=http://localhost:8082
export CORS_ALLOWED_ORIGINS=http://localhost:3000

Test with Port Forward

# Port forward to deployed gateway
kubectl port-forward -n gateway svc/gateway 8091:80

# Test health
curl http://localhost:8091/health

Related Resources

• Source Code: crates/gateway/
• Helm Chart: automation/helm/gateway/
• UI Fixtures: tests/fixtures/gateway_ui/
• Auth Playbooks: tests/fixtures/playbooks/api_integration/auth0/