diff options
| author |  Johannes Christ <[email protected]> | 2024-07-25 20:41:55 +0200 |
|---|---|---|
| committer | Johannes Christ <[email protected]> | 2024-07-25 20:56:43 +0200 |
| commit | c1551cbce8527beb07e8cb38bbcc1f57f4185210 (patch) | |
| tree | 2b5ff651bc9922af239c58c30d18d4aa40f63a4a /docs | |
| parent | Fix index page formatting (diff) | |
Update document titles
Diffstat (limited to 'docs')
23 files changed, 61 insertions, 61 deletions
diff --git a/docs/meeting_notes/2022-04-07.rst b/docs/meeting_notes/2022-04-07.rst index 21642d3..ee23a5d 100644 --- a/docs/meeting_notes/2022-04-07.rst +++ b/docs/meeting_notes/2022-04-07.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2022-04-07 +========== Agenda ------ diff --git a/docs/meeting_notes/2022-09-18.rst b/docs/meeting_notes/2022-09-18.rst index f6b56c2..163434c 100644 --- a/docs/meeting_notes/2022-09-18.rst +++ b/docs/meeting_notes/2022-09-18.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2022-09-18 +========== *Migrated from Notion*. diff --git a/docs/meeting_notes/2022-10-05.rst b/docs/meeting_notes/2022-10-05.rst index c405e01..e069299 100644 --- a/docs/meeting_notes/2022-10-05.rst +++ b/docs/meeting_notes/2022-10-05.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2022-10-05 +========== *Migrated from Notion*. diff --git a/docs/meeting_notes/2022-10-19.rst b/docs/meeting_notes/2022-10-19.rst index fa51d32..6de7f33 100644 --- a/docs/meeting_notes/2022-10-19.rst +++ b/docs/meeting_notes/2022-10-19.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2022-10-19 +========== *Migrated from Notion*. diff --git a/docs/meeting_notes/2022-10-26.rst b/docs/meeting_notes/2022-10-26.rst index 5684d7f..69f8c70 100644 --- a/docs/meeting_notes/2022-10-26.rst +++ b/docs/meeting_notes/2022-10-26.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2022-10-26 +========== *Migrated from Notion*. diff --git a/docs/meeting_notes/2022-11-02.rst b/docs/meeting_notes/2022-11-02.rst index 010b8f0..d9f415d 100644 --- a/docs/meeting_notes/2022-11-02.rst +++ b/docs/meeting_notes/2022-11-02.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2022-11-02 +========== *Migrated from Notion*. diff --git a/docs/meeting_notes/2022-11-23.rst b/docs/meeting_notes/2022-11-23.rst index 5f74fc6..19edd06 100644 --- a/docs/meeting_notes/2022-11-23.rst +++ b/docs/meeting_notes/2022-11-23.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2022-11-23 +========== *Migrated from Notion*. diff --git a/docs/meeting_notes/2023-02-08.rst b/docs/meeting_notes/2023-02-08.rst index c65193c..a161ba5 100644 --- a/docs/meeting_notes/2023-02-08.rst +++ b/docs/meeting_notes/2023-02-08.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2023-02-08 +========== *Migrated from Notion*. diff --git a/docs/meeting_notes/2023-02-21.rst b/docs/meeting_notes/2023-02-21.rst index c30c133..9de644c 100644 --- a/docs/meeting_notes/2023-02-21.rst +++ b/docs/meeting_notes/2023-02-21.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2023-02-21 +========== *Migrated from Notion*. diff --git a/docs/meeting_notes/2023-02-28.rst b/docs/meeting_notes/2023-02-28.rst index fe7dc47..1fb1093 100644 --- a/docs/meeting_notes/2023-02-28.rst +++ b/docs/meeting_notes/2023-02-28.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2023-02-28 +========== *Migrated from Notion*. diff --git a/docs/meeting_notes/2023-05-16.rst b/docs/meeting_notes/2023-05-16.rst index bafa941..79272a6 100644 --- a/docs/meeting_notes/2023-05-16.rst +++ b/docs/meeting_notes/2023-05-16.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2023-05-16 +========== *Migrated from Notion*. diff --git a/docs/meeting_notes/2023-07-11.rst b/docs/meeting_notes/2023-07-11.rst index 6c51f1c..68b1085 100644 --- a/docs/meeting_notes/2023-07-11.rst +++ b/docs/meeting_notes/2023-07-11.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2023-07-11 +========== Participants ------------ diff --git a/docs/meeting_notes/2023-07-18.rst b/docs/meeting_notes/2023-07-18.rst index 28f6c88..f37b2dc 100644 --- a/docs/meeting_notes/2023-07-18.rst +++ b/docs/meeting_notes/2023-07-18.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2023-07-18 +========== Secret management improvements ------------------------------ diff --git a/docs/meeting_notes/2023-07-25.rst b/docs/meeting_notes/2023-07-25.rst index be4d20c..0a3204c 100644 --- a/docs/meeting_notes/2023-07-25.rst +++ b/docs/meeting_notes/2023-07-25.rst @@ -1,4 +1,4 @@ -DevOps Meeting Notes -==================== +2023-07-25 +========== -Postponed to next week due to absence. +Postponed to next week due to Joe having a severe bellyache. diff --git a/docs/meeting_notes/2023-08-01.rst b/docs/meeting_notes/2023-08-01.rst index 925417a..67e4ee1 100644 --- a/docs/meeting_notes/2023-08-01.rst +++ b/docs/meeting_notes/2023-08-01.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2023-08-01 +========== Agenda ------ diff --git a/docs/meeting_notes/2023-08-08.rst b/docs/meeting_notes/2023-08-08.rst index 4b06d5e..0082cd3 100644 --- a/docs/meeting_notes/2023-08-08.rst +++ b/docs/meeting_notes/2023-08-08.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2023-08-08 +========== Agenda ------ diff --git a/docs/meeting_notes/2023-08-22.rst b/docs/meeting_notes/2023-08-22.rst index 67f53e9..a8d1287 100644 --- a/docs/meeting_notes/2023-08-22.rst +++ b/docs/meeting_notes/2023-08-22.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2023-08-22 +========== .. raw:: html diff --git a/docs/meeting_notes/2023-08-29.rst b/docs/meeting_notes/2023-08-29.rst index 8e0a7d4..da49c1e 100644 --- a/docs/meeting_notes/2023-08-29.rst +++ b/docs/meeting_notes/2023-08-29.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2023-08-29 +========== .. raw:: html diff --git a/docs/meeting_notes/2023-09-05.rst b/docs/meeting_notes/2023-09-05.rst index 2c80c2e..7556ab6 100644 --- a/docs/meeting_notes/2023-09-05.rst +++ b/docs/meeting_notes/2023-09-05.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2023-09-05 +========== .. raw:: html diff --git a/docs/meeting_notes/2023-09-12.rst b/docs/meeting_notes/2023-09-12.rst index 7bfcd1a..6dbb7c8 100644 --- a/docs/meeting_notes/2023-09-12.rst +++ b/docs/meeting_notes/2023-09-12.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2023-09-12 +========== .. raw:: html diff --git a/docs/meeting_notes/2024-07-02.rst b/docs/meeting_notes/2024-07-02.rst index 029d53e..4d2ba03 100644 --- a/docs/meeting_notes/2024-07-02.rst +++ b/docs/meeting_notes/2024-07-02.rst @@ -1,5 +1,5 @@ -DevOps Meeting Notes -==================== +2024-07-02 +========== .. raw:: html diff --git a/docs/postmortems/2021-01-12-site-cpu-ram-exhaustion.rst b/docs/postmortems/2021-01-12-site-cpu-ram-exhaustion.rst index 57f9fd8..f621782 100644 --- a/docs/postmortems/2021-01-12-site-cpu-ram-exhaustion.rst +++ b/docs/postmortems/2021-01-12-site-cpu-ram-exhaustion.rst @@ -5,7 +5,7 @@ At 03:01 UTC on Tuesday 12th January we experienced a momentary outage of our PostgreSQL database, causing some very minor service downtime. β οΈ Leadup -========= +--------- *List the sequence of events that led to the incident* @@ -35,7 +35,7 @@ before 3:40 where no statistics were reported is the actual outage period where the Kubernetes node had some networking errors. π₯ Impact -========= +--------- *Describe how internal and external users were impacted during the incident* @@ -48,7 +48,7 @@ Most services stayed up that did not depend on PostgreSQL, and the site remained stable after the sync had been cancelled. ποΈ Detection -============ +--------------- *Report when the team detected the incident, and how we could improve detection time* @@ -69,7 +69,7 @@ time of crashing by examining the logs which pointed us towards the user sync process. ππΏββοΈ Response -================ +----------------------- *Who responded to the incident, and what obstacles did they encounter?* @@ -78,7 +78,7 @@ encountered other than the node being less performant than we would like due to the CPU starvation. ππ½ββοΈ Recovery -================ +--------------------------- *How was the incident resolved? How can we improve future mitigation?* @@ -113,7 +113,7 @@ as you can see site hit this twice (during the periods where it was trying to sync 80k users at once) π Five Whyβs -============= +--------------------------- *Run a 5-whys analysis to understand the true cause of the incident.* @@ -125,7 +125,7 @@ trying to sync 80k users at once) resulting in 80k users needing updating. π± Blameless root cause -======================= +----------------------- *Note the final root cause and describe what needs to change to prevent reoccurrance* @@ -137,14 +137,14 @@ See the follow up tasks on exactly how we can avoid this in future, itβs a relatively easy mitigation. π€ Lessons learned -================== +----------------------- *What did we learn from this incident?* - Django (or DRF) does not like huge update queries. βοΈ Follow-up tasks -================== +------------------ *List any tasks we should complete that are relevant to this incident* diff --git a/docs/postmortems/2021-07-11-cascading-node-failures.rst b/docs/postmortems/2021-07-11-cascading-node-failures.rst index 6cd30f3..b2e5cdf 100644 --- a/docs/postmortems/2021-07-11-cascading-node-failures.rst +++ b/docs/postmortems/2021-07-11-cascading-node-failures.rst @@ -7,7 +7,7 @@ affected node volumes were placed into a state where they could not be mounted. β οΈ Leadup -========= +---------- *List the sequence of events that led to the incident* @@ -32,7 +32,7 @@ mounted. service was not healthy. π₯ Impact -========= +---------- *Describe how internal and external users were impacted during the incident* @@ -48,7 +48,7 @@ PostgreSQL was restored early on so for the most part Moderation could continue. ποΈ Detection -============ +--------------- *Report when the team detected the incident, and how we could improve detection time* @@ -61,7 +61,7 @@ conn delta vs.Β conn total), but for the most part alerting time was satisfactory here. ππΏββοΈ Response -================ +----------------- *Who responded to the incident, and what obstacles did they encounter?* @@ -70,7 +70,7 @@ at Linode to remount the affected volumes, a support ticket has been created. ππ½ββοΈ Recovery -================ +------------------ *How was the incident resolved? How can we improve future mitigation?* @@ -262,7 +262,7 @@ Joe </details> π Five Whyβs -============= +--------------- *Run a 5-whys analysis to understand the true cause of the incident.* @@ -305,7 +305,7 @@ have. I presume itβs some sort of worker race condition, but Iβve not been able to reproduce it. π± Blameless root cause -======================= +----------------------- *Note the final root cause and describe what needs to change to prevent reoccurrence* @@ -315,7 +315,7 @@ mounting was difficult, it took multiple hours for this to be resolved since we had to wait for the volumes to unlock so they could be cloned. π€ Lessons learned -================== +------------------ *What did we learn from this incident?* @@ -325,7 +325,7 @@ We need to look at why Django is doing this and mitigations of the fault to prevent this from occurring again. βοΈ Follow-up tasks -================== +------------------ *List any tasks we should complete that are relevant to this incident* |