Merge #2408: Scaffold server config via a bootstrapping script

Refactor configuration into a pydantic-based python constants file,
and add a utility to auto-populate guild data.

Squashed commits:
* use basic config for demo purposes

* fix guiding comments

* update var names for proper context reflection

* fix wront iteration var

* add all roles, & channels

* load categories

* separate sections in env file

* ignore .env.server

* rename change_log to changelog

This also adds a default env file to look for

* remove instantiation of webhooks

* add most of the default configs

These will mostly be fetched from the .env.default file, which won't be bootstrapped

* warn when categories/roles/channels are not found

* add env file to keep server defaults

* fix malformatted value in the .env.default

* add default server env variables

* update the sections formatting in default env file

* fallback to server env when loading constants

* add guild basic defaults

* update change_log channel name to changelog

* add the Guid settings prefix

* make _Guild inherit from EnvConfig

* add webhook defaults

* add python_news defaults to the server env

* ad missing webhooks prefix

* update bootstrapper logger name

* update priority of the env loaded files

According to Pydantic's docs: "Later files in the list/tuple will take priority over earlier files."

* warn user that default value from PyDis' config will be used

* add colours default config

* add antispam config

* update antispam references

* add redis default cfg

* add Stats, Cooldowns and CleanMessages consts

This also includes their default values

* add Metabase to constants

This also includes its default values

* add URLS to constants

This also includes its default values

* use the Field class to provide defaults

This avoids overriding & changing the `fields` of the `Config` class "dynamically"

* add keys constant class

* add Guild conf

* replace dash with underscore in script

* appease linter

* transform attributes of AntiSpam to dict when needed

This ensures that the application stays backwards compatible

* add root_validator for the colours class

This enables the conversion from hex to int easily since it's not a supported type by pydantic

* reinstate the role & channels combinations

* rename URLS to URLs

* add emojis & icons constants

* add filter constants & their default values

* remove all useless spaces

* instantiate the keys class

* add bot prefix to default env file

* fetch Bot constants from env vars instead of the prefix ones

* add Miscellaneous config

* instantiate poor forgotten Miscellaneous config

* add final touches to the constants module

This includes removing dups, adding missing channels & fixing type casts

* move all default values to constants.py

This is done by using the `Field` class.

It allows us to
1. Set defaults, in case the variables are not configured
2. Load them from a env variable under a specific name (for backwards comp)
3. load it from any env variable file that contains the right prefix

* ignore all .env files

* load BOT_TOKEN & GUILD_ID from .env

* allow _GUILD to read its id from the `GUILD_ID` env var

* base Webhooks settings off of a Webhook model

* create necessary webhooks if non existent

* appease flake8 docstrings error

* make the script idempotent

* update type hints

* uppercase all consts

* make webhook channel optional

* add httpx to its own dependency group

This group will be optional & only related to the bootstrapper

* replace requests with httpx

* pass client as param

* include raise_for_status as a response hook

* rename get_webhook to webhook_exists

* update docstring of the constants module

* use "." as a separator

* update script to account for already created webhooks

* make ANTI_SPAM_RULES a module level constant

This ensures that flake8 doesn't complain about making a function call in the function's signature

* remove the manual resolving of .env paths

* update usages of AntiSpam constants

* remove forgotten assignment of rule_config

* remove useless assignments of env file names

* delete default config-default.yml

* update docstrings of CodeBlockCog to reference constants.py

* add a poetry task that runs the bootstrapping script

* add python-dotenv to the config-bootstrap group

* update hook name to _raise_for_status

* construct site_api in _URLs

* remove __name__ == '__main__'guard

* Revert "construct site_api in _URLs"

This reverts commit 1c555c4280c6a0bdd452319cbd3ffcd0370f5d48.

* remove usage of the Field class

* update env var keys that the bootstrapping script needs

* use API_KEYS.SITE_API as env var in docker compose instead of BOT_API_KEY

* use basic config for demo purposes

* fix guiding comments

* update var names for proper context reflection

* fix wront iteration var

* add all roles, & channels

* load categories

* separate sections in env file

* ignore .env.server

* rename change_log to changelog

This also adds a default env file to look for

* remove instantiation of webhooks

* add most of the default configs

These will mostly be fetched from the .env.default file, which won't be bootstrapped

* warn when categories/roles/channels are not found

* add env file to keep server defaults

* fix malformatted value in the .env.default

* add default server env variables

* update the sections formatting in default env file

* fallback to server env when loading constants

* add guild basic defaults

* update change_log channel name to changelog

* add the Guid settings prefix

* make _Guild inherit from EnvConfig

* add webhook defaults

* add python_news defaults to the server env

* ad missing webhooks prefix

* update bootstrapper logger name

* update priority of the env loaded files

According to Pydantic's docs: "Later files in the list/tuple will take priority over earlier files."

* warn user that default value from PyDis' config will be used

* add colours default config

* add antispam config

* update antispam references

* add redis default cfg

* add Stats, Cooldowns and CleanMessages consts

This also includes their default values

* add Metabase to constants

This also includes its default values

* add URLS to constants

This also includes its default values

* use the Field class to provide defaults

This avoids overriding & changing the `fields` of the `Config` class "dynamically"

* add keys constant class

* add Guild conf

* replace dash with underscore in script

* appease linter

* transform attributes of AntiSpam to dict when needed

This ensures that the application stays backwards compatible

* add root_validator for the colours class

This enables the conversion from hex to int easily since it's not a supported type by pydantic

* reinstate the role & channels combinations

* rename URLS to URLs

* add emojis & icons constants

* add filter constants & their default values

* remove all useless spaces

* instantiate the keys class

* add bot prefix to default env file

* fetch Bot constants from env vars instead of the prefix ones

* add Miscellaneous config

* instantiate poor forgotten Miscellaneous config

* add final touches to the constants module

This includes removing dups, adding missing channels & fixing type casts

* move all default values to constants.py

This is done by using the `Field` class.

It allows us to
1. Set defaults, in case the variables are not configured
2. Load them from a env variable under a specific name (for backwards comp)
3. load it from any env variable file that contains the right prefix

* ignore all .env files

* load BOT_TOKEN & GUILD_ID from .env

* allow _GUILD to read its id from the `GUILD_ID` env var

* base Webhooks settings off of a Webhook model

* create necessary webhooks if non existent

* appease flake8 docstrings error

* make the script idempotent

* update type hints

* uppercase all consts

* make webhook channel optional

* add httpx to its own dependency group

This group will be optional & only related to the bootstrapper

* replace requests with httpx

* pass client as param

* include raise_for_status as a response hook

* rename get_webhook to webhook_exists

* update docstring of the constants module

* use "." as a separator

* update script to account for already created webhooks

* make ANTI_SPAM_RULES a module level constant

This ensures that flake8 doesn't complain about making a function call in the function's signature

* remove the manual resolving of .env paths

* update usages of AntiSpam constants

* remove forgotten assignment of rule_config

* remove useless assignments of env file names

* delete default config-default.yml

* update docstrings of CodeBlockCog to reference constants.py

* add a poetry task that runs the bootstrapping script

* add python-dotenv to the config-bootstrap group

* update hook name to _raise_for_status

* construct site_api in _URLs

* remove __name__ == '__main__'guard

* Revert "construct site_api in _URLs"

This reverts commit 1c555c4280c6a0bdd452319cbd3ffcd0370f5d48.

* remove usage of the Field class

* update env var keys that the bootstrapping script needs

* use API_KEYS.SITE_API as env var in docker compose instead of BOT_API_KEY

* relock dependencies

* update snekbox's defaults

* add support for ot channels

* rename help_system_forum to python_help

* rename nomination_archive to nomination_voting_archive

* rename appeals2 to appeals_2

* yeet sprinters role out

* rename all big_brother_logs instances to big_brother

The purpose is to adhere to what we have in prod

* rename bootstrap_config.py to botstrap.py

* update module name of the configure poetry task

* update error messages to reflect the new keys needed for env variables

* install dotenv as an extra with pydantic

* update all prefixes to "_" (underscore)

* log tuple of (channel_name, channel_id) in the config verifier

* update needed default values for docker compose env var

* relock dependencies

* update forgotten delimiters & env prefixes

1 files changed, 2 insertions, 2 deletions

diff --git a/pyproject.toml b/pyproject.toml
index 67f72e776..227ea4303 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -11,6 +11,7 @@ python = "3.10.*"
 # See https://bot-core.pythondiscord.com/ for docs.
 pydis_core = { version = "9.5.1", extras = ["async-rediscache"] }
 
+aiohttp = "3.8.3"
 arrow = "1.2.3"
 beautifulsoup4 = "4.11.2"
 colorama = { version = "0.4.6", markers = "sys_platform == 'win32'" }
@@ -28,7 +29,7 @@ rapidfuzz = "2.13.7"
 regex = "2022.10.31"
 sentry-sdk = "1.16.0"
 tldextract = "3.4.0"
-pydantic = "1.10.5"
+pydantic = { version = "1.10.5", extras = ["dotenv"]}
 
 [tool.poetry.dev-dependencies]
 coverage = "7.2.1"
@@ -45,7 +46,6 @@ pre-commit = "3.1.1"
 pip-licenses = "4.1.0"
 pytest = "7.2.2"
 pytest-cov = "4.0.0"
-python-dotenv = "1.0.0"
 pytest-subtests = "0.10.0"
 pytest-xdist = "3.2.0"
 taskipy = "1.10.3"