Triton and TensorRT-LLM

Serving custom models with Triton Inference Server and optimised TensorRT-LLM engines.

1. 01

   Overview

   When to use Triton/TensorRT-LLM versus a prebuilt NIM.

   Coming soon
2. 02

   Triton on OpenShift

   Deploy Triton Inference Server and serve a custom ONNX or PyTorch model.

   Coming soon
3. 03

   TensorRT-LLM

   Build an optimised engine for an open LLM and compare latency and throughput with plain vLLM.

   Coming soon
4. 04

   Model ensembles

   Chain pre-processing, model and post-processing in one Triton pipeline.

   Coming soon