Skupper Model Provider — VAN Topology

Skupper Model Provider — VAN Topology A architecture diagram generated by Archify. localhost (podman) rhtevan-work (LAN, podman) rhel-ai (AWS, podman) OpenShift CRC (kubernetes) Local Client · curl / Goose / LiteLLM · Architecture component Local Client curl / Goose / LiteLLM Listener :10000 · model-api-rhtevan-work · localhost (podman) Listener :10000 model-api-rhtevan-work Listener :9000 · model-api-rhel-ai · localhost (podman) Listener :9000 model-api-rhel-ai Skupper Router · local-ezhang (podman) · localhost (podman) Skupper Router local-ezhang (podman) RouterAccess :55672 · inbound from CRC · localhost (podman) RouterAccess :55672 inbound from CRC Controller · system-controller · localhost (podman) Controller system-controller Skupper Router · hub-rhtevan-work · rhtevan-work (LAN, podman) Skupper Router hub-rhtevan-work Controller · rhtevan-work · rhtevan-work (LAN, podman) Controller rhtevan-work Granite 3B · g3b-16k (llama.cpp) · rhtevan-work (LAN, podman) Granite 3B g3b-16k (llama.cpp) Skupper Router · hub-rhel-ai · rhel-ai (AWS, podman) Skupper Router hub-rhel-ai Controller · rhel-ai · rhel-ai (AWS, podman) Controller rhel-ai Granite 8B FP8 · g8b-fp8-spec-128k (vLLM) · rhel-ai (AWS, podman) Granite 8B FP8 g8b-fp8-spec-128k (vLLM) Skupper Router · crc-ezhang (k8s) · OpenShift CRC (kubernetes) Skupper Router crc-ezhang (k8s) Listener :9000 · model-listener-rhel-ai · OpenShift CRC (kubernetes) Listener :9000 model-listener-rhel-ai Listener :10000 · model-listener-rhtevan-work · OpenShift CRC (kubernetes) Listener :10000 model-listener-rhtevan-work HTTP :10000 HTTP :9000 routing key routing key AMQPS :55671 AMQPS :8000 connector connector manage manage manage AMQPS :8000 AMQPS :55672 route route accept Legend Backend Cloud Security External

Consumer Sites

  • • Localhost: dual listeners (:9000, :10000) + RouterAccess :55672
  • • CRC: 2 links (rhel-ai direct, local-ezhang 2-hop), 2 listeners
  • • Outbound interior links to hub routers

Provider Sites

  • • rhtevan-work: Granite 3B Q4_K_M (LAN, port 55671)
  • • rhel-ai: Granite 8B FP8 + spec decoding (AWS, port 8000)
  • • Model containers managed by hosted-model-ctl

Resilience

  • • CRC reaches rhtevan-work via 2-hop if rhel-ai is down
  • • Auto-restart via systemd on crash
  • • Partial availability: skip unreachable, serve available