Jomablue - Notice history

During an update to a subset of customer instances, a configuration error in the deployment process caused some environments to become temporarily unavailable. This resulted in affected instances being unable to load or operate as expected.

Once the issue was identified, we halted the rollout and began restoring each impacted instance. Most environments were corrected quickly and all were fully restored.

Our deployment process requires multiple automated and manual human verification steps to avoid situations like this, in this particular case - the third party interface we use to orchestrate deployments was misrepresenting the deployment configuration. We’ve since updated our deployment process to revoke and completely avoid this configuration issue from occurring again, and we’ve added additional monitoring that will alert us immediately if a similar condition is ever detected.

This incident has been resolved. A post mortem will be provided here in due course.

We have addressed the cause and are currently monitoring the result. More details regarding the incident will be published in due course.

We are continuing to resolve the incident, services are restoring.

Our upstream SMS provider temporarily paused outbound SMS delivery after detecting unusual traffic affecting a subset of US numbers. As a result, SMS messages sent from Jomablue were not processed during this period.

Once we became aware of the interruption, we rotated all SMS-related credentials as a precaution and worked with our provider to resume normal operation. SMS sending was unavailable while this rotation was in progress, but service has since been fully restored.

This incident has been resolved. A post mortem will be made available here in due course.

We had addressed the cause, SMS sending has resumed and we are currently monitoring the result. We will provide more information in due course.

We have identified the cause and are bringing SMS services back online.