Reducto Status

We identified and mitigated an issue affecting a subset of inference requests, which resulted in intermittent 5xx errors. The impacted services were restarted and capacity was restored. The platform is stable, and all systems are operating normally.

Tue

Elevated endpoint latency

5:54 PM

Services should now be fully restored.

Thu

Degraded performance on parse with use_checkboxes

8:47 PM

recovered - a portion of parse requests from 1330-1343 PST failed, all operational now

Tue

Degraded performance on ocr_system = legacy

11:41 PM

All services functional

Fri

Queue saturation in US processing

3:25 PM

Resolved queue saturation incident.

Wed

Elevated failure rate with upstream inference provider

3:53 PM

All endpoints should now be working as expected. All upstream provider issues are now resolved.

May

Sat

Scheduled Database Maintenance

8:30 PM

Maintenance has completed

Thu

Elevated Failure Rate

5:15 PM

Fully mitigated.

Wed

Rolling deploy caused transient failures

8:38 PM

Fully resolved.

Tue

Degraded Split and Extract Endpoints

3:38 PM

all systems operational

Fri

Investigating degraded performance on Reducto's API

4:03 AM

We are seeing full recovery.

Wed

Investigating Elevated Error Rates

11:24 PM

Resolved.

Rolling deploy caused transient errors

10:00 PM

Changes were rolled back and error rate dropped to normal rates.

April

No incidents reported