Write-up published
Read our write-up of this incident at https://bem-team.notion.site/AWS-us-east-1-Outage-295e8f8e1d7c8026a9f1dff2f07011d4?pvs=74
Resolved
Textract issues recover after AWS team’s successful fixes shipped to smooth over the NLB
Monitoring
Textract timeout issues emerge again due to AWS team’s updates to the AWS Network Load Balancer (NLB)
Monitoring
Textract issues recover
Monitoring
DNS recovers
Textract timeout issues continue, leading to degraded performance on some inputs processed through bem
Identified
DNS outage with app.bem.ai and api.bem.ai unreachable by many clients with inconsistent DNS resolution
AWS Textract time out issues begin