Resolved
All inference services restored. New jobs will continue to succeed.
Monitoring
An updated deployment has restored service for all new jobs.
Identified
The root cause was identified and the inference change was reverted.
Investigating
A small percentage of jobs errored out due to elevated errors with our inference services