Ben Chan

Give your fallback models one name with virtual models

Save an ordered list of models as a virtual model, send its name as the model, and AI Gateway tries each model in turn until one answers.

We integrated Coraza into our Traffic Policy system, ran it against every request to ngrok.com, and blocked 1.2% of traffic. Here's how we designed it and what attack patterns we're seeing.

Ben Chan

•

~2.1K words

•Topics:

• Engineering