From 0db34fc876a678b5f18244e7314a66790a1a741b Mon Sep 17 00:00:00 2001 From: sco1 Date: Wed, 3 Apr 2019 23:00:53 -0400 Subject: Update contributor doc --- CONTRIBUTING.md | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 57 insertions(+), 4 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ab3d695..429fbf1 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -19,9 +19,10 @@ Note that contributions may be rejected on the basis of a contributor failing to 5. **Avoid frequent pushes to the main repository**. This goes for PRs opened against your fork as well. Our test build pipelines are triggered every time a push to the repository (or PR) is made. Try to batch your commits until you've finished working for that session, or you've reached a point where collaborators need your commits to continue their own work. This also provides you the opportunity to amend commits for minor changes rather than having to commit them on their own because you've already pushed. * This includes merging master into your branch. Try to leave merging from master for after your PR passes review; a maintainer will bring your PR up to date before merging. Exceptions to this include: resolving merge conflicts, needing something that was pushed to master for your branch, or something was pushed to master that could potentionally affect the functionality of what you're writing. 6. **Don't fight the framework**. Every framework has its flaws, but the frameworks we've picked out have been carefully chosen for their particular merits. If you can avoid it, please resist reimplementing swathes of framework logic - the work has already been done for you! -7. If someone is working on a pull request, **do not open your own pull request for the same task**. Instead, collaborate with the author(s) of the existing pull request. Communication is key, and there's no point in two separate implementations of the same thing. +7. If someone is working on an issue or pull request, **do not open your own pull request for the same task**. Instead, collaborate with the author(s) of the existing pull request. Duplicate PRs opened without communicating with the other author(s) and/or PyDis staff will be closed. Communication is key, and there's no point in two separate implementations of the same thing. * One option is to fork the other contributor's repository and submit your changes to their branch with your own pull request. We suggest following these guidelines when interacting with their repository as well. -8. **Work as a team** and collaborate whereever possible. Keep things friendly and help each other out - these are shared projects and nobody likes to have their feet trodden on. + * The author(s) of inactive PRs and claimed issues will be be pinged after a week of inactivity for an update. Continued inactivity may result in the issue being released back to the community and/or PR closure. +8. **Work as a team** and collaborate wherever possible. Keep things friendly and help each other out - these are shared projects and nobody likes to have their feet trodden on. 9. **Internal projects are internal**. As a contributor, you have access to information that the rest of the server does not. With this trust comes responsibility - do not release any information you have learned as a result of your contributor position. We are very strict about announcing things at specific times, and many staff members will not appreciate a disruption of the announcement schedule. 10. All changes to snekbox **must be validated against the bot** before PRing. Please don't leave this for reviewers to discover. Guides for setting up the developer environments can be [found below](#developer-environment). @@ -39,15 +40,67 @@ A working environment for the [PyDis site](https://github.com/python-discord/sit * [Site](https://wiki.pythondiscord.com/wiki/contributing/project/site) * [Bot](https://wiki.pythondiscord.com/wiki/contributing/project/bot) -### Logging levels +When pulling down changes from GitHub, remember to sync your environment using `pipenv sync --dev` to ensure you're using the most up-to-date versions the project's dependencies. + +### Type Hinting +[PEP 484](https://www.python.org/dev/peps/pep-0484/) formally specifies type hints for Python functions, added to the Python Standard Library in version 3.5. Type hints are recognized by most modern code editing tools and provide useful insight into both the input and output types of a function, preventing the user from having to go through the codebase to determine these types. + +For example: + +```py +def foo(input_1: int, input_2: dict) -> bool: +``` + +Tells us that `foo` accepts an `int` and a `dict` and returns a `bool`. + +All function declarations should be type hinted in code contributed to the PyDis organization. + +For more information, see *[PEP 483](https://www.python.org/dev/peps/pep-0483/) - The Theory of Type Hints* and Python's documentation for the [`typing`](https://docs.python.org/3/library/typing.html) module. + +### AutoDoc Formatting Directives +Many documentation packages provide support for automatic documentation generation from the codebase's docstrings. These tools utilize special formatting directives to enable richer formatting in the generated documentation. + +For example: + +```py +def foo(bar: int, baz: dict=None) -> bool: + """ + Does some things with some stuff. + + :param bar: Some input + :param baz: Optional, some other input + + :return: Some boolean + """ +``` + +Since PyDis does not utilize automatic documentation generation, use of this syntax should not be used in code contributed to the organization. Should the purpose and type of the input variables not be easily discernable from the variable name and type annotation, a prose explanation can be used. Explicit references to variables, functions, classes, etc. should be wrapped with backticks (`` ` ``). + +For example, the above docstring would become: + +```py +def foo(bar: int, baz: dict=None) -> bool: + """ + Does some things with some stuff. + + This function takes an index, `bar` and checks for its presence in the database `baz`, passed as a dictionary. Returns `False` if `baz` is not passed. + """ +``` + +### Logging Levels The project currently defines [`logging`](https://docs.python.org/3/library/logging.html) levels as follows: -* **TRACE:** Use this for tracing every step of a complex process. That way we can see which step of the process failed. Err on the side of verbose. **Note:** This is a PyDis-implemented logging level. * **DEBUG:** Someone is interacting with the application, and the application is behaving as expected. * **INFO:** Something completely ordinary happened. Like a cog loading during startup. * **WARNING:** Someone is interacting with the application in an unexpected way or the application is responding in an unexpected way, but without causing an error. * **ERROR:** An error that affects the specific part that is being interacted with * **CRITICAL:** An error that affects the whole application. +### Work in Progress (WIP) PRs +Github [has introduced a new PR feature](https://github.blog/2019-02-14-introducing-draft-pull-requests/) that allows the PR author to mark it as a WIP. This provides both a visual and functional indicator that the contents of the PR are in a draft state and not yet ready for formal review. + +This feature should be utilized in place of the traditional method of prepending `[WIP]` to the PR title. + ## Footnotes This document was inspired by the [Glowstone contribution guidelines](https://github.com/GlowstoneMC/Glowstone/blob/dev/docs/CONTRIBUTING.md). + -- cgit v1.2.3 From 58fe2057c4d7ebcc90116b973a07d0d8cddc5cf0 Mon Sep 17 00:00:00 2001 From: sco1 Date: Sun, 21 Apr 2019 16:32:49 -0400 Subject: Manually pin urllib3 version to mitigate vulnerability --- Pipfile | 1 + Pipfile.lock | 83 ++++++++++++++++++++++++++++-------------------------------- 2 files changed, 39 insertions(+), 45 deletions(-) diff --git a/Pipfile b/Pipfile index 3e0174d..678a3fd 100644 --- a/Pipfile +++ b/Pipfile @@ -6,6 +6,7 @@ name = "pypi" [packages] pika = "*" docker = "*" +urllib3 = ">=1.24.2,<1.25" [dev-packages] flask = "*" diff --git a/Pipfile.lock b/Pipfile.lock index 11a6201..262131c 100644 --- a/Pipfile.lock +++ b/Pipfile.lock @@ -1,7 +1,7 @@ { "_meta": { "hash": { - "sha256": "0c682a13bfe227f8f95421c69f8d68b6fca2d4a841a45688f2cac078a97b1db1" + "sha256": "22c5372766fde7766ebb92d523cfc1eb0192bf1cc2b4860c51b50937998da40d" }, "pipfile-spec": 6, "requires": { @@ -32,11 +32,11 @@ }, "docker": { "hashes": [ - "sha256:0076504c42b6a671c8e7c252913f59852669f5f882522f4d320ec7613b853553", - "sha256:d2c14d2cc7d54818897cc6f3cf73923c4e7dfe12f08f7bddda9dbea7fa82ea36" + "sha256:2b1f48041cfdcc9f6b5da0e04e0e326ded225e736762ade2060000e708f4c9b7", + "sha256:c456ded5420af5860441219ff8e51cdec531d65f4a9e948ccd4133e063b72f50" ], "index": "pypi", - "version": "==3.7.1" + "version": "==3.7.2" }, "docker-pycreds": { "hashes": [ @@ -54,11 +54,11 @@ }, "pika": { "hashes": [ - "sha256:b0640085f1d6398fd47bb16a17713053e26578192821ea5d928772b8e6a28789", - "sha256:b785e0d5f74a94781bd7d020862eb137d2b56cef2a21475aadbe5bcc8ec4db15" + "sha256:0c50285f00a8b4816f2c9a44469107d9e738ba3a90386f14b625d8cceef4f6ae", + "sha256:5ba83d3daffccb92788d24facdab62a3db6aa03b8a6d709b03dc792d35c0dfe8" ], "index": "pypi", - "version": "==0.13.1" + "version": "==1.0.1" }, "requests": { "hashes": [ @@ -76,10 +76,11 @@ }, "urllib3": { "hashes": [ - "sha256:61bf29cada3fc2fbefad4fdf059ea4bd1b4a86d2b6d15e1c7c0b582b9752fe39", - "sha256:de9529817c93f27c8ccbfead6985011db27bd0ddfcdb2d86f3f663385c6a9c22" + "sha256:4c291ca23bbb55c76518905869ef34bdd5f0e46af7afe6861e8375643ffee1a0", + "sha256:9a247273df709c4fedb38c711e44292304f73f39ab01beda9f6b9fc375669ac3" ], - "version": "==1.24.1" + "index": "pypi", + "version": "==1.24.2" }, "websocket-client": { "hashes": [ @@ -113,10 +114,10 @@ }, "cfgv": { "hashes": [ - "sha256:39f8475d8eca48639f900daffa3f8bd2f60a31d989df41a9f81c5ad1779a66eb", - "sha256:a6a4366d32799a6bfb6f577ebe113b27ba8d1bae43cb57133b1472c1c3dae227" + "sha256:6e9f2feea5e84bc71e56abd703140d7a2c250fc5ba38b8702fd6a68ed4e3b2ef", + "sha256:e7f186d4a36c099a9e20b04ac3108bd8bb9b9257e692ce18c8c3764d5cb12172" ], - "version": "==1.5.0" + "version": "==1.6.0" }, "click": { "hashes": [ @@ -178,11 +179,11 @@ }, "flake8-bugbear": { "hashes": [ - "sha256:07b6e769d7f4e168d590f7088eae40f6ddd9fa4952bed31602def65842682c83", - "sha256:0ccf56975f4db1d69dc1cf3598c99d768ebf95d0cad27d76087954aa399b515a" + "sha256:5070774b668be92c4312e5ca82748ddf4ecaa7a24ff062662681bb745c7896eb", + "sha256:fef9c9826d14ec23187ae1edeb3c6513c4e46bf0e70d86bac38f7d9aabae113d" ], "index": "pypi", - "version": "==18.8.0" + "version": "==19.3.0" }, "flake8-docstrings": { "hashes": [ @@ -323,25 +324,17 @@ }, "identify": { "hashes": [ - "sha256:407cbb36e8b72b45cfa96a97ae13ccabca4c36557e03616958bd895dfcd3f77d", - "sha256:721abbbb1269fa1172799119981c22c5ace022544ce82eedc29b1b0d753baaa5" + "sha256:244e7864ef59f0c7c50c6db73f58564151d91345cd9b76ed793458953578cadd", + "sha256:8ff062f90ad4b09cfe79b5dfb7a12e40f19d2e68a5c9598a49be45f16aba7171" ], - "version": "==1.4.0" + "version": "==1.4.1" }, "importlib-metadata": { "hashes": [ - "sha256:a17ce1a8c7bff1e8674cb12c992375d8d0800c9190177ecf0ad93e0097224095", - "sha256:b50191ead8c70adfa12495fba19ce6d75f2e0275c14c5a7beb653d6799b512bd" + "sha256:46fc60c34b6ed7547e2a723fc8de6dc2e3a1173f8423246b3ce497f064e9c3de", + "sha256:bc136180e961875af88b1ab85b4009f4f1278f8396a60526c0009f503a1a96ca" ], - "version": "==0.8" - }, - "importlib-resources": { - "hashes": [ - "sha256:6e2783b2538bd5a14678284a3962b0660c715e5a0f10243fd5e00a4b5974f50b", - "sha256:d3279fd0f6f847cced9f7acc19bd3e5df54d34f93a2e7bb5f238f81545787078" - ], - "markers": "python_version < '3.7'", - "version": "==1.0.2" + "version": "==0.9" }, "itsdangerous": { "hashes": [ @@ -352,10 +345,10 @@ }, "jinja2": { "hashes": [ - "sha256:74c935a1b8bb9a3947c50a54766a969d4846290e1e788ea44c1392163723c3bd", - "sha256:f84be1bb0040caca4cea721fcbbbbd61f9be9464ca236387158b0feea01914a4" + "sha256:065c4f02ebe7f7cf559e49ee5a95fb800a9e4528727aec6f24402a5374c65013", + "sha256:14dd6caf1527abb21f08f86c784eac40853ba93edb79552aa1e4b8aef1b61c7b" ], - "version": "==2.10" + "version": "==2.10.1" }, "junit-xml": { "hashes": [ @@ -405,11 +398,11 @@ }, "more-itertools": { "hashes": [ - "sha256:0125e8f60e9e031347105eb1682cef932f5e97d7b9a1a28d9bf00c22a5daef40", - "sha256:590044e3942351a1bdb1de960b739ff4ce277960f2425ad4509446dbace8d9d1" + "sha256:2112d2ca570bb7c3e53ea1a35cd5df42bb0fd10c45f0fb97178679c3c03d64c7", + "sha256:c3e4748ba1aad8dba30a4886b0b1a2004f9a863837b8654e7059eebf727afa5a" ], "markers": "python_version > '2.7'", - "version": "==6.0.0" + "version": "==7.0.0" }, "nodeenv": { "hashes": [ @@ -426,11 +419,11 @@ }, "pre-commit": { "hashes": [ - "sha256:d3d69c63ae7b7584c4b51446b0b583d454548f9df92575b2fe93a68ec800c4d3", - "sha256:fc512f129b9526e35e80d656a16a31c198f584c4fce3a5c739045b5140584917" + "sha256:2576a2776098f3902ef9540a84696e8e06bf18a337ce43a6a889e7fa5d26c4c5", + "sha256:82f2f2d657d7f9280de9f927ae56886d60b9ef7f3714eae92d12713cd9cb9e11" ], "index": "pypi", - "version": "==1.14.4" + "version": "==1.15.2" }, "py": { "hashes": [ @@ -463,11 +456,11 @@ }, "pytest": { "hashes": [ - "sha256:592eaa2c33fae68c7d75aacf042efc9f77b27c08a6224a4f59beab8d9a420523", - "sha256:ad3ad5c450284819ecde191a654c09b0ec72257a2c711b9633d677c71c9850c4" + "sha256:3773f4c235918987d51daf1db66d51c99fac654c81d6f2f709a046ab446d5e5d", + "sha256:b7802283b70ca24d7119b32915efa7c409982f59913c1a6c0640aacf118b95f5" ], "index": "pypi", - "version": "==4.3.1" + "version": "==4.4.1" }, "pytest-cov": { "hashes": [ @@ -530,10 +523,10 @@ }, "werkzeug": { "hashes": [ - "sha256:96da23fa8ccecbc3ae832a83df5c722c11547d021637faacb0bec4dd2f4666c8", - "sha256:ca5c2dcd367d6c0df87185b9082929d255358f5391923269335782b213d52655" + "sha256:0a73e8bb2ff2feecfc5d56e6f458f5b99290ef34f565ffb2665801ff7de6af7a", + "sha256:7fad9770a8778f9576693f0cc29c7dcc36964df916b83734f4431c0e612a7fbc" ], - "version": "==0.15.1" + "version": "==0.15.2" }, "zipp": { "hashes": [ -- cgit v1.2.3 From 3366ec3aeb25a800cbe3dbef5d410b807f9220d4 Mon Sep 17 00:00:00 2001 From: sco1 Date: Wed, 24 Apr 2019 17:25:40 -0400 Subject: Remove docstring newline linting from ignored codes Remove docstring-ends-with-period enforcement to allow for freedom of punctuation. Removes D202, D204, D400 --- .flake8 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.flake8 b/.flake8 index cc5f423..0e50ded 100644 --- a/.flake8 +++ b/.flake8 @@ -6,11 +6,11 @@ ignore= # Missing Docstrings D100,D104,D107, # Docstring Whitespace - D202,D203,D204,D212,D214,D215, + D203,D212,D214,D215, # Docstring Quotes D301,D302, # Docstring Content - D400,D401,D402,D405,D406,D407,D408,D409,D410,D411,D412,D413,D414 + D401,D402,D405,D406,D407,D408,D409,D410,D411,D412,D413,D414 exclude= __pycache__,.cache, venv,.venv, -- cgit v1.2.3 From 3a146c286344474bd5d6958b3c2eab10341ff399 Mon Sep 17 00:00:00 2001 From: sco1 Date: Wed, 24 Apr 2019 17:25:47 -0400 Subject: Relint Snekbox with new linting rules --- config.py | 1 - rmq.py | 2 -- snekbox.py | 3 --- snekweb.py | 2 -- 4 files changed, 8 deletions(-) diff --git a/config.py b/config.py index 5e4f648..5ca23bb 100644 --- a/config.py +++ b/config.py @@ -6,7 +6,6 @@ from docker.errors import NotFound def autodiscover(): """Search for the snekbox container and return its IPv4 address.""" - container_names = ["rmq", "pdrmq", "snekbox_pdrmq_1"] client = docker.from_env() diff --git a/rmq.py b/rmq.py index 919ef19..29fc448 100644 --- a/rmq.py +++ b/rmq.py @@ -25,7 +25,6 @@ class Rmq: def consume(self, queue=QUEUE, callback=None, thread_ws=None, run_once=False): """Subscribe to read from a RMQ channel.""" - while True: try: connection = pika.BlockingConnection(self.con_params) @@ -72,7 +71,6 @@ class Rmq: def publish(self, message, queue=QUEUE, routingkey=ROUTING_KEY, exchange=EXCHANGE): """Open a connection to publish (write) to a RMQ channel.""" - try: connection = pika.BlockingConnection(self.con_params) diff --git a/snekbox.py b/snekbox.py index f8d7c31..7491672 100644 --- a/snekbox.py +++ b/snekbox.py @@ -45,7 +45,6 @@ class Snekbox: Returns the output of executing the command (stdout) if successful, or a error message if the execution failed. """ - args = [self.nsjail_binary, '-Mo', '--rlimit_as', '700', '--chroot', '/', @@ -108,7 +107,6 @@ class Snekbox: to RMQ. Once published, the system exits, since the snekboxes are created and disposed of per-execution. """ - msg = body.decode('utf-8') result = '' snek_msg = json.loads(msg) @@ -125,7 +123,6 @@ class Snekbox: def message_handler(self, ch, method, properties, body, thread_ws=None): """Spawns a daemon process that handles RMQ messages.""" - p = multiprocessing.Process(target=self.execute, args=(body,)) p.daemon = True p.start() diff --git a/snekweb.py b/snekweb.py index ff1a72c..3e20fda 100644 --- a/snekweb.py +++ b/snekweb.py @@ -22,14 +22,12 @@ log = app.logger @app.route('/') def index(): """Root path returns standard index.html.""" - return render_template('index.html') @sockets.route('/ws/') def websocket_route(ws, snekboxid): """Opens a websocket that spawns and connects to a snekbox daemon.""" - localdata = threading.local() localdata.thread_ws = ws -- cgit v1.2.3 From 341de95b6e7ebf2205248bd69cce7510c8e9ecfe Mon Sep 17 00:00:00 2001 From: sco1 Date: Wed, 24 Apr 2019 17:28:43 -0400 Subject: Put back D400 as ignored code Oops --- .flake8 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.flake8 b/.flake8 index 0e50ded..13bd00b 100644 --- a/.flake8 +++ b/.flake8 @@ -10,7 +10,7 @@ ignore= # Docstring Quotes D301,D302, # Docstring Content - D401,D402,D405,D406,D407,D408,D409,D410,D411,D412,D413,D414 + D400, D401,D402,D405,D406,D407,D408,D409,D410,D411,D412,D413,D414 exclude= __pycache__,.cache, venv,.venv, -- cgit v1.2.3 From 505dc2d18835e492057f00df2b67daa890f4585e Mon Sep 17 00:00:00 2001 From: Mark Date: Wed, 24 Apr 2019 18:43:42 -0700 Subject: Remove space between ignored flake8 rules --- .flake8 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.flake8 b/.flake8 index 13bd00b..e3a35a8 100644 --- a/.flake8 +++ b/.flake8 @@ -10,7 +10,7 @@ ignore= # Docstring Quotes D301,D302, # Docstring Content - D400, D401,D402,D405,D406,D407,D408,D409,D410,D411,D412,D413,D414 + D400,D401,D402,D405,D406,D407,D408,D409,D410,D411,D412,D413,D414 exclude= __pycache__,.cache, venv,.venv, -- cgit v1.2.3