Impact

At least five users experienced issues when attempting to access certain organizations on the platform. The issue started on UTC-5 24-10-30 09:53 and was proactively discovered 14.4 minutes (TTD) later by a staff member who reported through our help desk [1] that they could not access the organizations. The problem was resolved in 2.4 hours (TTF) resulting in a total impact of 2.8 hours (TTR) [2].

Cause

The issue was caused by enabling a feature in the database that automatically deletes records based on a set expiration time. When users accepted invitations to join an organization, the system did not clear the expiration_time field, leaving past timestamps on some records. This unintentionally deleted organization access records with expired timestamps [3].

Solution

The issue was addressed by updating the system to automatically clear the expiration_time field when an invitation is accepted, preventing active organization access from being deleted. A migration was also implemented to correct existing organization_access records with past expiration times, ensuring they would not be mistakenly removed. Additionally, a migration was added to restore items that had already been deleted due to the expiration policy [4].

Conclusion

Additional checks were added in the testing process to verify that the expiration_time field updates correctly after an invitation is accepted, ensuring the issue does not reoccur [5]. NCOMPLETE_PERSPECTIVE < MISSING_TEST