diff options
| author |  Johannes Christ <[email protected]> | 2024-07-24 20:09:42 +0200 |
|---|---|---|
| committer | Johannes Christ <[email protected]> | 2024-07-25 20:06:54 +0200 |
| commit | a4d7e92d544aeb43dbe1fcd8648d97e0dbf7b9d3 (patch) | |
| tree | 183318852234388654c99514e45f095af8c21676 /docs/meeting_notes | |
| parent | Add link to DevOps Kanban board in meeting template (#420) (diff) | |
Improve documentation
This commit ports our documentation to Sphinx.
The reason for this is straightforward. We need to improve both the
quality and the accessibility of our documentation. Hugo is not capable
of doing this, as its primary output format is HTML. Sphinx builds
plenty of high-quality output formats out of the box, and incentivizes
writing good documentation.
Diffstat (limited to 'docs/meeting_notes')
23 files changed, 951 insertions, 0 deletions
diff --git a/docs/meeting_notes/2022-04-07.rst b/docs/meeting_notes/2022-04-07.rst new file mode 100644 index 0000000..21642d3 --- /dev/null +++ b/docs/meeting_notes/2022-04-07.rst @@ -0,0 +1,20 @@ +DevOps Meeting Notes +==================== + +Agenda +------ + +- No updates, as last week’s meeting did not take place + +Roadmap review & planning +------------------------- + +What are we working on for the next meeting? + +- Help wanted for #57 (h-asgi) +- #58 (postgres exporter) needs a new review +- #54 (firewall in VPN) will be done by Johannes +- We need a testing environment #67 +- Johannes will add a Graphite role #31 +- Sofi will take a look at #29 +- #41 (policy bot) will be taken care of by Johannes diff --git a/docs/meeting_notes/2022-09-18.rst b/docs/meeting_notes/2022-09-18.rst new file mode 100644 index 0000000..f6b56c2 --- /dev/null +++ b/docs/meeting_notes/2022-09-18.rst @@ -0,0 +1,74 @@ +DevOps Meeting Notes +==================== + +*Migrated from Notion*. + +Agenda +------ + +- Joe will grant Chris access to the netcup hosts. + +NetKube status +~~~~~~~~~~~~~~ + +- **Rollout** + + - ☒ RBAC configuration and access granting + - ☒ Most nodes are enrolled, Joe will re-check + - ``turing``, ``ritchie``, ``lovelace`` and ``neumann`` will be + Kubernetes nodes + - ``hopper`` will be the storage server + +- **Storage drivers** + + - Not needed, everything that needs persistent storage will run on + hopper + - Netcup does not support storage resize + - We can download more RAM if we need it + - A couple of services still need volume mounts: Ghost, Grafana & + Graphite + +- **Control plane high availability** + + - Joe mentions that in the case the control plane dies, everything + else will die as well + - If the control plane in Germany dies, so will Johannes + +- **Early plans for migration** + + - We can use the Ansible repository issues for a good schedule + - Hopper runs ``nginx`` + - Statement from Joe: > “There is an nginx ingress running on every + node in the cluster, okay, > okay? We don’t, the way that’s, + that’s as a service is a NodePort, right? > So it has a normal IP, + but the port will be like a random port in the range > of the + 30,000s. Remember that? Hold on. Is he writing rude nodes? And + then… > We have nginx, so this is where it’s like a little bit, + like, not nice, I > guess we just like, cronjob it, to pull the + nodes, like, every minute or > so, and then update the config if + they change. But then it’s just like… > nginx is like a catalogue + of nodes. Wahhh, you drive me crazy.” + + - “Nah, it makes sense!” + + - “It does!” + + - Joe will figure this out with assistance from his voices. + +Open authentication +~~~~~~~~~~~~~~~~~~~ + +- Joe and Johannes will check out OpenLDAP as a JumpCloud alternative + starting from this evening +- Sofi has experience with OpenLDAP + +Sponsorship +----------- + +This meeting has been sponsored by Chris Hemsworth Lovering’s +relationship therapy company, “Love To Love By Lovering”. You can sign +up by sending a mail to [email protected]. + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2022-10-05.rst b/docs/meeting_notes/2022-10-05.rst new file mode 100644 index 0000000..c405e01 --- /dev/null +++ b/docs/meeting_notes/2022-10-05.rst @@ -0,0 +1,13 @@ +DevOps Meeting Notes +==================== + +*Migrated from Notion*. + +Agenda +------ + +- Joe Banks configured proper RBAC for Chris, Johannes and Joe himself + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2022-10-19.rst b/docs/meeting_notes/2022-10-19.rst new file mode 100644 index 0000000..fa51d32 --- /dev/null +++ b/docs/meeting_notes/2022-10-19.rst @@ -0,0 +1,31 @@ +DevOps Meeting Notes +==================== + +*Migrated from Notion*. + +Agenda +------ + +- One hour of gartic phone, for team spirit. +- Created user accounts for Sofi and Hassan +- Joe created an architecture diagram of the NGINX setup + + - *This is still in Notion* + +- Joe explained his NGINX plans: > “It’s not actually that hard, right? + So you spawn 5 instances of nginx in a > DaemonSet, because then one + gets deployed to every node okay, following? > Then we get NodePort, + instead of LoadBalancers or whatever, which will get > a random port + allocatead in the 35000 range, and that will go to nginx, and > on + each of those ports, it will go to nginx, right? And then we poll the + > Kubernetes API and what is the port that each of these nginx + instances is > running on, and add that into a roundrobin on the + fifth node. Right? Yeah. > That’s correct. That won’t do TLS though, + so that will just HAProxy. Yeah.” +- Joe will terminate our JumpCloud account +- Chris reset the Minecraft server +- Email alerting needs to be configured + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2022-10-26.rst b/docs/meeting_notes/2022-10-26.rst new file mode 100644 index 0000000..5684d7f --- /dev/null +++ b/docs/meeting_notes/2022-10-26.rst @@ -0,0 +1,18 @@ +DevOps Meeting Notes +==================== + +*Migrated from Notion*. + +Agenda +------ + +- Chris upgraded PostgreSQL to 15 in production +- Johannes added the Kubernetes user creation script into the + Kubernetes repository in the docs + +*(The rest of the meeting was discussion about the NetKube setup, which +has been scrapped since)*. + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2022-11-02.rst b/docs/meeting_notes/2022-11-02.rst new file mode 100644 index 0000000..010b8f0 --- /dev/null +++ b/docs/meeting_notes/2022-11-02.rst @@ -0,0 +1,27 @@ +DevOps Meeting Notes +==================== + +*Migrated from Notion*. + +Agenda +------ + +Hanging behaviour of ModMail +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- `Source <https://discord.com/channels/267624335836053506/675756741417369640/1036720683067134052>`__ + +- Maybe use `Signals + a + debugger <https://stackoverflow.com/a/25329467>`__? + +- … using `something like pdb for the + debugger <https://wiki.python.org/moin/PythonDebuggingTools>`__? + +- Or `GDB, as it seems handy to poke at stuck multi-threaded python + software <https://wiki.python.org/moin/DebuggingWithGdb>`__? + +- ModMail has been upgraded to version 4 + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2022-11-23.rst b/docs/meeting_notes/2022-11-23.rst new file mode 100644 index 0000000..5f74fc6 --- /dev/null +++ b/docs/meeting_notes/2022-11-23.rst @@ -0,0 +1,30 @@ +DevOps Meeting Notes +==================== + +*Migrated from Notion*. + +Agenda +------ + +*(This meeting was mostly about NetKube, with the following strange text +included, and everything outside of the text has been removed since the +NetKube plans have been scrapped)*. + +Joe Banks, after a month-long hiatus to become a dad to every second +girl on uni campus, has managed to pull up to the DevOps meeting. + +We are considering using Kubespray (https://kubespray.io/#/) in order to +deploy a production-ready bare-metal Kubernetes cluster without +involvement from Joe “Busy With Poly Girlfriend #20” Banks. + +At the moment cluster networking is not working and Joe mentions that +the last time he has touched it, it worked perfectly fine. However, the +last time he touched it there was only 1 node, and therefore no +inter-node communications. + +Joe thinks he remembers installing 3 nodes, however, we at the DevOps +team believe this to be a marijuana dream + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2023-02-08.rst b/docs/meeting_notes/2023-02-08.rst new file mode 100644 index 0000000..c65193c --- /dev/null +++ b/docs/meeting_notes/2023-02-08.rst @@ -0,0 +1,17 @@ +DevOps Meeting Notes +==================== + +*Migrated from Notion*. + +Agenda +------ + +- Investigation into deploying a VPN tool such as WireGuard to have + inter-node communication between the Netcup hosts. + +*(The rest of this meeting was mostly about NetKube, which has since +been scrapped)*. + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2023-02-21.rst b/docs/meeting_notes/2023-02-21.rst new file mode 100644 index 0000000..c30c133 --- /dev/null +++ b/docs/meeting_notes/2023-02-21.rst @@ -0,0 +1,31 @@ +DevOps Meeting Notes +==================== + +*Migrated from Notion*. + +Agenda +------ + +Reusable status embed workflows +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Further discussion with Bella followed +- Upstream pull request can be found at + `python-discord/bot#2400 <https://github.com/python-discord/bot/pull/2400>`__ + +Local vagrant testing setup +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Our new `testing setup using Vagrant + VMs <https://github.com/python-discord/infra/pull/78>`__ has been + merged. + +A visit from Mina +~~~~~~~~~~~~~~~~~ + +Mina checked in to make sure we’re operating at peak Volkswagen-like +efficiency. + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2023-02-28.rst b/docs/meeting_notes/2023-02-28.rst new file mode 100644 index 0000000..fe7dc47 --- /dev/null +++ b/docs/meeting_notes/2023-02-28.rst @@ -0,0 +1,16 @@ +DevOps Meeting Notes +==================== + +*Migrated from Notion*. + +Agenda +------ + +- Black knight’s CI & dependabot configuration has been mirrored across + all important repositories + +- The test server has been updated for the new configuration + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2023-05-16.rst b/docs/meeting_notes/2023-05-16.rst new file mode 100644 index 0000000..bafa941 --- /dev/null +++ b/docs/meeting_notes/2023-05-16.rst @@ -0,0 +1,15 @@ +DevOps Meeting Notes +==================== + +*Migrated from Notion*. + +Agenda +------ + +- Bella set up `CI bot docker image + build <https://github.com/python-discord/bot/pull/2603>`__ to make + sure that wheels are available. + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2023-07-11.rst b/docs/meeting_notes/2023-07-11.rst new file mode 100644 index 0000000..6c51f1c --- /dev/null +++ b/docs/meeting_notes/2023-07-11.rst @@ -0,0 +1,41 @@ +DevOps Meeting Notes +==================== + +Participants +------------ + +- Chris, Johannes, Bella, Bradley + +Agenda +------ + +New Ansible setup +~~~~~~~~~~~~~~~~~ + +Chris presented the new Ansible setup he’s been working on. We plan to +use WireGuard for networking. We agreed that selfhosting Kubernetes is +not the way to go. In general, the main benefit from switching away to +Linode to Netcup is going to be a ton more resources from the Netcup +root servers we were given. The original issue with Linode’s AKS of +constantly having problems with volumes has not been present for a +while. Chris mentions the one remaining issue is that we’re at half our +memory capacity just at idle. + +It’s our decision where to go from here - we can stick to the Kubernetes +setup or decide on migrating to the Ansible setup. But we have bare +metal access to the Netcup hosts, which makes e.g. managing databases a +lot easier. Chris mentions the possibility to only use Netcup for our +persistence and Linode AKS for anything else, but this has the issue of +us relying on two sponsors for our infrastructure instead of one. + +PostgreSQL was set up to run on ``lovelace``. + +Decision +~~~~~~~~ + +**It was decided to hold a vote on the core development channel, which +will be evaluated next week to see how to proceed with the setup**. + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2023-07-18.rst b/docs/meeting_notes/2023-07-18.rst new file mode 100644 index 0000000..28f6c88 --- /dev/null +++ b/docs/meeting_notes/2023-07-18.rst @@ -0,0 +1,42 @@ +DevOps Meeting Notes +==================== + +Secret management improvements +------------------------------ + +To allow for **better management of our Kubernetes secrets**, Chris set +out to configure ``git-crypt`` in GPG key mode. For comparison, the +previous approach was that secrets were stored in Kubernetes only and +had to be accessed via ``kubectl``, and now ``git-crypt`` allows us to +transparently work with the files in unencrypted manner locally, whilst +having them secure on the remote, all via ``.gitattributes``. + +The following people currently have access to this: + +- Johannes Christ [email protected] + (``8C05D0E98B7914EDEBDCC8CC8E8E09282F2E17AF``) +- Chris Lovering [email protected] + (``1DA91E6CE87E3C1FCE32BC0CB6ED85CC5872D5E4``) +- Joe Banks [email protected] (``509CDFFC2D0783A33CF87D2B703EE21DE4D4D9C9``) + +For Hassan, we are still waiting on response regarding his GPG key +accuracy. + +The pull request for the work can be found `at +python-discord/kubernetes#156 <https://github.com/python-discord/kubernetes/pull/156>`__. + +**To have your key added, please contact any of the existing key +holders**. More documentation on this topic is pending to be written, +see +`python-discord/kubernetes#157 <https://github.com/python-discord/kubernetes/issues/157>`__. + +Infrastructure migration decision +--------------------------------- + +The voting started `last week <./2023-07-11.md>`__ will be properly +talked about `next week <./2023-07-25.md>`__, so far it looks like we’re +definitely not selfhosting Kubernetes at the very least. + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2023-07-25.rst b/docs/meeting_notes/2023-07-25.rst new file mode 100644 index 0000000..be4d20c --- /dev/null +++ b/docs/meeting_notes/2023-07-25.rst @@ -0,0 +1,4 @@ +DevOps Meeting Notes +==================== + +Postponed to next week due to absence. diff --git a/docs/meeting_notes/2023-08-01.rst b/docs/meeting_notes/2023-08-01.rst new file mode 100644 index 0000000..925417a --- /dev/null +++ b/docs/meeting_notes/2023-08-01.rst @@ -0,0 +1,66 @@ +DevOps Meeting Notes +==================== + +Agenda +------ + +Infrastructure migration +~~~~~~~~~~~~~~~~~~~~~~~~ + +The vote is tied. Chris and Johannes decided that we should test out +migrating the PostgreSQL database at the very least. We then have more +freedom about our data. What we need to do: + +- Allow PostgreSQL connections from LKE’s static IPs in the firewall +- Whitelist the static IPs from Linode via ``pg_hba.conf`` +- Schedule downtime for the PostgreSQL database +- **At downtime** + + - Take writers offline + - Dump database from Linode into Netcup + - Update all the client’s database URLs to point to netcup + - Restart writers + +We want to rely on the restore to create everything properly, but will +need to test run this beforehand. The following ``pg_virtualenv`` +command has showcased that it works properly: + +.. code:: sh + + kubectl exec -it postgres-... -- pg_dumpall -U pythondiscord \ + | pg_virtualenv psql -v ON_ERROR_STOP=1 + +Note however that the database extension ``pg_repack`` needs to be +installed. + +Before we can get started, we need to allow the PostgreSQL role to +configure ``pg_hba.conf`` and ``postgresql.conf`` entries. + +Meeting notes +~~~~~~~~~~~~~ + +We’re using GitHub at the moment. Some are left in Notion. We should +migrate these to GitHub to have a uniform interface: Johannes will pick +up +`python-discord/infra#108 <https://github.com/python-discord/infra/issues/108>`__ +to merge them together into Git, as its more open than Notion. + +Ansible lint failures in the infra repository +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Excluding the vault was found as the working solution here, as +implemented by Chris. + +Kubernetes repository pull requests +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +These were cleaned up thanks to Chris. + +Roadmap review & planning +------------------------- + +- Chris will prepare the PostgreSQL configuration mentioned above. + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2023-08-08.rst b/docs/meeting_notes/2023-08-08.rst new file mode 100644 index 0000000..4b06d5e --- /dev/null +++ b/docs/meeting_notes/2023-08-08.rst @@ -0,0 +1,54 @@ +DevOps Meeting Notes +==================== + +Agenda +------ + +- Configuration of PostgreSQL and the PostgreSQL exporter + + - **No time so far**. Chris has been busy with renovating his living + room, and Johannes has been busy with renovating his bedroom. + Bradley prefers to remain quiet. + + - Chris will try to work on this in the coming week and will try to + have Bella around as well, since he wanted to join the setup. + +- **Potential slot for GPG key signing of DevOps members**. External + verification will be necessary. + + - Skipped. No webcam on Chris. + +- We need to assign a **librarian** to keep our documents organized + according to a system. Johannes is happy to do this for now. + + - Let’s move the existing documentation from the Kubernetes + repository into the infra repository. See + `kubernetes#161 <https://github.com/python-discord/kubernetes/issues/161>`__. + + - **Our Notion DevOps space is full of junk**. Outside of that, it’s + not open to read for outside contributors, and does not leave much + choice over which client to use for editing content. + + - Chris agrees, without looking on it - just from memory. We + should move it to the infra repository. (The meeting notes have + already been transferred). + + - Bella suggests to add some automation to make keeping everything + in clean order less tedious. + +- We may want to integrate the **Kubernetes repository** and the infra + repository together altogether, however there are a lot of + repositories referencing the deployment manifests that would need to + be updated. + + - Chris mentions that regardless of what we do, we should - at the + very least move all documentation into the ``infra`` repository, + including the static site generator. At the moment we’re using + Jekyll but we’re open to trying alternatives such as Hugo. + +- We closed some issues and pull requests in the repositories for late + spring cleaning. + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2 autoindent conceallevel=2: --> diff --git a/docs/meeting_notes/2023-08-22.rst b/docs/meeting_notes/2023-08-22.rst new file mode 100644 index 0000000..67f53e9 --- /dev/null +++ b/docs/meeting_notes/2023-08-22.rst @@ -0,0 +1,40 @@ +DevOps Meeting Notes +==================== + +.. raw:: html + + <!-- + + Useful links + + - Infra open issues: https://github.com/python-discord/infra/issues + + - infra open pull requests: https://github.com/python-discord/infra/pulls + + - *If* any open issue or pull request needs discussion, why was the existing + asynchronous logged communication over GitHub insufficient? + + --> + +Agenda +------ + +- Bella said he is on the streets. **We should start a gofundme**. + + - After some more conversation this just means he is on vacation and + currently taking a walk. + +- Chris has been busy with turning his living room into a picasso art + collection, Johannes has been busy with renovating his bedroom, and + Bella is not home. + + - Our next priority is winning. + +- We checked out some issues with documentation generation in + ``bot-core`` that Bella has mentioned. We managed to fix one issue + with pydantic by adding it to an exclude list but ran into another + problem next. + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2023-08-29.rst b/docs/meeting_notes/2023-08-29.rst new file mode 100644 index 0000000..8e0a7d4 --- /dev/null +++ b/docs/meeting_notes/2023-08-29.rst @@ -0,0 +1,65 @@ +DevOps Meeting Notes +==================== + +.. raw:: html + + <!-- + + Useful links + + - Infra open issues: https://github.com/python-discord/infra/issues + + - infra open pull requests: https://github.com/python-discord/infra/pulls + + - *If* any open issue or pull request needs discussion, why was the existing + asynchronous logged communication over GitHub insufficient? + + --> + +Agenda +------ + +- **Bella is still on the streets** + + - The Python Discord Bella On The Streets Fundraising Campaign Q3 + 2023 has not been successful so far. To help Bella receive French + citizenship, Joe has put up a French flag behind himself in the + meeting. + + - Joe corrects my sarcasm. It is an Italian flag, not a French + flag. The reason for this flag is that his new prime interest + on campus was born in Italy. + +- **The SnekBox CI build is pretty slow** + + - Guix and Nix are not alternatives. Neither is Ubuntu + + - We use pyenv to build multiple Python versions for a new feature + + - The feature is not rolled out yet + + - Part of the problem is that we build twice in the ``build`` and + the ``deploy`` stage + + - On rollout, Joe tested it and it works fine + +- No update on the Hugo build yet + +- For snowflake, Johannes will write a proposal to the admins for + hosting it + + - We should consider talking about the following points: + + - statistically ~8% of Tor traffic is problematic (10% of traffic + is to hidden services, 80% of hidden service traffic is for + illegal services) + + - overall the project’s position and our ideal is to help people + for a good cause + + - all traffic is forwarded to the Tor network, the service is + lightweight and only proxies encrypted traffic there + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2023-09-05.rst b/docs/meeting_notes/2023-09-05.rst new file mode 100644 index 0000000..2c80c2e --- /dev/null +++ b/docs/meeting_notes/2023-09-05.rst @@ -0,0 +1,53 @@ +DevOps Meeting Notes +==================== + +.. raw:: html + + <!-- + + Useful links + + - Infra open issues: https://github.com/python-discord/infra/issues + + - infra open pull requests: https://github.com/python-discord/infra/pulls + + - *If* any open issue or pull request needs discussion, why was the existing + asynchronous logged communication over GitHub insufficient? + + --> + +Agenda +------ + +- No update on the Hugo build yet + +- Johannes wrote a proposal for snowflake proxy to be deployed to our + netcup hosts + + - Admins discussed and came to the conclusion that since we don’t + own the servers, we got the servers from netcup as a sponsorship + to host our infra, so using them to host something that isn’t our + infra doesn’t seem right. + +- Lots of dependabot PRs closed + + - https://github.com/search?q=org%3Apython-discord++is%3Apr+is%3Aopen+label%3A%22area%3A+dependencies%22&type=pullrequests&ref=advsearch + - Closed ~50% of PRs + +- Workers repo has had its CI rewritten, all workers have consistent + package.json, scripts, and using the new style of cloudflare workers + which don’t use webpack + +- Metricity updated to SQLAlchemy 2 + +- Olli CI PR is up + + - https://github.com/python-discord/olli/pull/25 + +- Sir-Robin pydantic constants PR is up + + - https://github.com/python-discord/sir-robin/pull/93 + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2023-09-12.rst b/docs/meeting_notes/2023-09-12.rst new file mode 100644 index 0000000..7bfcd1a --- /dev/null +++ b/docs/meeting_notes/2023-09-12.rst @@ -0,0 +1,73 @@ +DevOps Meeting Notes +==================== + +.. raw:: html + + <!-- + + Useful links + + - Infra open issues: https://github.com/python-discord/infra/issues + + - infra open pull requests: https://github.com/python-discord/infra/pulls + + - *If* any open issue or pull request needs discussion, why was the existing + asynchronous logged communication over GitHub insufficient? + + --> + +Agenda +------ + +- We have reason to believe that Bella is still on the streets. Worse, + Bella is not available at the moment, leading us to believe that + Bella has still not found a home. + + - Eight minutes into the meeting, Bella joins, complaining about the + bad internet. He mentions he is still on the streets (this may + contribute to the bad internet factor). + +- Chris made Mina leave with his repeated comments about Bella being + homeless, reminding Mina of the growing unemployment rate within the + DevOps team. As head of HR she cannot further support this matter. + +- About #139, Bella mentions that online websites may cover the same + need that we have, but it may not be really useful for having it as a + command. + + - Chris adds that “if someone wants to do it, I don’t mind” and “I + don’t think it would be very useful for a command, but I think it + would be fun to learn for someone implementing it”. As long as + whoever is implementing is is aware that it would not be used too + much, it would be fine. + +- No progress on the hugo front + +- Our email service with workers will be forward only + + - With postfix you will be able to reply. Joe wants to have an + excuse to play with Cloudflare workers though. + +- `50 open pull requests from + dependabot <https://github.com/search?q=org%3Apython-discord++is%3Apr+is%3Aopen+author%3Aapp%2Fdependabot&type=pullrequests&ref=advsearch>`__ + + - Tip from The Man: press ^D to make a bookmark in your browser + + - “Those can just be blindly merged” - Chris + +- Grouping of dependencies: Dependabot now allows you to group together + multiple dependency updates into a single pull request. + + - Possible approaches suggested: Group all the docker updates + together, group any linting dependencies together (would just + require a big RegEx). Dependabot natively works with its own + dependency groups here (e.g. Docker, Pip). + +- Mr. Hemlock wants to raise his roof: It’s his project for this + Autumn. We, the team, are looking forward to his project - especially + Bella, who is currently looking for housing. “It’s all coming + together”, said Chris to the situation. + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/2024-07-02.rst b/docs/meeting_notes/2024-07-02.rst new file mode 100644 index 0000000..029d53e --- /dev/null +++ b/docs/meeting_notes/2024-07-02.rst @@ -0,0 +1,171 @@ +DevOps Meeting Notes +==================== + +.. raw:: html + + <!-- + + Useful links + + - Infra open issues: https://github.com/python-discord/infra/issues + + - infra open pull requests: https://github.com/python-discord/infra/pulls + + - *If* any open issue or pull request needs discussion, why was the existing + asynchronous logged communication over GitHub insufficient? + + --> + +Attendees +--------- + +Joe and Johannes. + +Chris unfortunately died in a fatal train accident and could not attend +the meeting. This incident will be rectified in the next release, +“Lovering 2.0: Immortability”. + +Bella is out on the streets again. We are waiting for approval from the +Python Discord admins to run another fundraiser. + +Agenda +------ + +- **Configuration of renovate** (Joe) + + We are replacing dependabot with renovatebot. Johannes welcomes this + decision. Joe says we are looking for automatic deployment from + Kubernetes to make sure that any updates are automatically deployed. + + **Conclusion**: Implemented. + +- **Resizing Netcup servers** (Joe, Johannes) + + We can probably get rid of turing, assess what else we want to deploy + on lovelace, and then ask for a resize. + + **Conclusion**: Create issue to move things off turing, remove it + from the inventory, remove it from documentation, power it off, then + have Joe ask for server removal. + +- **Updating the public statistics page** (Johannes) + + Discussing and showcasing possible alternatives to the current + infrastructure powering https://stats.pythondiscord.com via the + https://github.com/python-discord/public-stats repository. Johannes + presents his current scripts that cuddle RRDTool into loading data + out of metricity, Joe says we will discuss with Chris what to do + here. + + The likely way going forward will be that *we will open an issue to + set it up*, the setup will contain an Ansible role to deploy the + cronjob and the script onto lovelace alongside with the ``rrdtool`` + PostgreSQL user. + + **Conclusion**: Johannes will create an issue and codify the setup in + Ansible. + +- **New blog powered by Hugo** (Johannes) + + Our current Ghost-powered blog is a tiny bit strange, and the + onboarding ramp to contribute articles is large. We want to migrate + this to Hugo - Johannes is leading the effort on it. The main work + will be building an appropriate theme, as no nicely suitable + replacement theme has been found so far. Front-end contributors would + be nice for this, although currently everything is still local on my + machine. + + Joe mentions that we don’t need to take anything particularly similar + to the current Ghost theme, just some vague resemblance would be + nice. Most of the recommended Hugo themes would probably work. + Johannes will check it out further. + + **Conclusion**: Try the `hugo-casper-two + theme <https://github.com/eueung/hugo-casper-two>`__ and report back. + +- **Finger server** (Joe, Johannes) + + Joe recently proposed `the deployment of a finger + server <https://github.com/python-discord/infra/pull/373>`__. Do we + want this and if yes, how are we going to proceed with this? If we do + not want any, running the ``pinky`` command locally or via ``ssh`` + would be a sound idea. We also need to consider whether members will + update their files regularly - we may want to incorporate + functionality for this into e.g. King Arthur. + + Joe says that we shouldn’t put a lot of development effort into it, + it would be simply a novelty thing. + + **Conclusion**: This is a nice cheap win for some fun which should + just be a simple Python file (via Twisted’s Finger protocol support + or whatever) that connects to LDAP (see Keycloak authentication + server) and outputs information. We could possibly integrate this + into King Arthur as well, so the querying workflow could look like KA + -> fingerd -> LDAP, or people could use finger commands directly. + +- **Keycloak authentication server** (Joe) + + Joe mentions that we are deploying a Keycloak server because for some + members authenticating via GitHub is cumbersome, for instance because + their GitHub account is connected to their employer’s GitHub + Enterprise installation. We could hook up a finger server to the LDAP + endpoint. Joe also mentions that we might want to set up e-mail + forwarding from pydis addresses to users via the user database that + will be stored in Keycloak. + + Currently we only have a Keycloak installation that stores items in + PostgreSQL. This installation can federate to LDAP - we would simply + have to settle on some directory service backend. Joe suggests + FreeIPA because he’s familar with it (including the Keycloak + integration). The problem is that it doesn’t work on Debian. The + alternative proposal, given that we’re saving ~50$/month on Linode, + would be spinning up a Rocky VM with FreeIPA on it on Linode (we + already have the budget) or ask Netcup for another VM. Ultimately, + the system to run FreeIPA would be something CentOS-based. One aspect + to consider is networking security: in Linode we could use their + private cloud endpoint feature to securely expose the LDAP server to + Keycloak and other services in Kubernetes, if we were to run it in + Netcup, we would need to use a similar setup to what we currently + have with PostgreSQL. + + Any Python Discord user would be managed in LDAP, and Keycloak has + the necessary roles to write back into LDAP. Keeping the users in + FreeIPA up-to-date would be a somewhat manual procedure. Joe’s plan + was to pick up the user’s Discord username and use + ``[email protected]`` as their name and do account setup as part of + the staff onboarding. + + **Conclusion**: Will wait for Chris to discuss this further, but we + simply need to decide where we want to run the LDAP service. + +- **Flux CD** (Joe) + + Joe proposes deploying `flux <https://fluxcd.io/>`__ as a way to + improve the way we manage our CI/CD. We want the cluster to be able + to synchronize its state with the git repository. There are some + manifests in the repository currently that are not in sync with the + cluster version. + + **Conclusion**: Approved, Joe will create an issue and do it. + +- **Polonium** (Chris) + + Question came up regarding why the bot does not write to the database + directly. Joe said it’s not perfect to have the bot write to it + directly - in metricity it works but it’s not perfect. Chris probably + had good reason: separation of intent. + + **Conclusion**: Approved, write to R&D for financing. + +- **Rethinking Bella: Suggested measures to gain autonomy** (Chris) + + Chris will present our current plans to biologically re-think and + improve Bella’s current architecture by means of + hypertrophy-supported capillary enlargements, with the final goal of + gaining complete control and ownership over the World Economic Forum + by 2026. As Bella is currently on parental leave, we will send him + the result of this voting via NNCP. + +.. raw:: html + + <!-- vim: set textwidth=80 sw=2 ts=2: --> diff --git a/docs/meeting_notes/index.rst b/docs/meeting_notes/index.rst new file mode 100644 index 0000000..cf7bb14 --- /dev/null +++ b/docs/meeting_notes/index.rst @@ -0,0 +1,31 @@ +Meeting notes +============= + +Minutes for previous Devops meetings. + +.. toctree:: + :maxdepth: 2 + :caption: Contents: + + template + 2022-04-07 + 2022-09-18 + 2022-10-05 + 2022-10-19 + 2022-10-26 + 2022-11-02 + 2022-11-23 + 2023-02-08 + 2023-02-21 + 2023-02-28 + 2023-05-16 + 2023-07-11 + 2023-07-18 + 2023-07-25 + 2023-08-01 + 2023-08-08 + 2023-08-22 + 2023-08-29 + 2023-09-05 + 2023-09-12 + 2024-07-02 diff --git a/docs/meeting_notes/template.rst b/docs/meeting_notes/template.rst new file mode 100644 index 0000000..e8bc719 --- /dev/null +++ b/docs/meeting_notes/template.rst @@ -0,0 +1,19 @@ +DevOps Meeting Notes +==================== + +.. + Useful links + + - Infra Kanban board: https://github.com/orgs/python-discord/projects/17/views/4 + + - Infra open issues: https://github.com/python-discord/infra/issues + + - infra open pull requests: https://github.com/python-discord/infra/pulls + + - *If* any open issue or pull request needs discussion, why was the existing + asynchronous logged communication over GitHub insufficient? + +Agenda +------ + +.. vim: set textwidth=80 sw=2 ts=2: |