aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeLines
...
| * Install pep8-namingGravatar MarkKoz2021-01-09-1/+17
| | | | | | | | | | | | It is a flake8 plugin which enforces PEP 8 naming conventions. Resolves #63
| * Update dev dependenciesGravatar MarkKoz2021-01-09-11/+11
|/
* Create dependency cache directory if not presentGravatar Joe Banks2021-01-10-1/+1
|
* Remove erroneous slash from dependencies listGravatar Joe Banks2021-01-10-1/+1
|
* Add leading slash to post-start commandGravatar Joe Banks2021-01-10-1/+1
|
* Merge pull request #87 from python-discord/feat/deps/site-isolationGravatar Joe Banks2021-01-10-446/+149
|\
| * CI: install numpyGravatar MarkKoz2021-01-09-1/+2
| | | | | | | | One of the unit tests depends on numpy.
| * Prevent setting GIT_SHA from invalidating the Docker layer cacheGravatar MarkKoz2021-01-09-2/+5
| |
| * Document how to install additional packagesGravatar MarkKoz2021-01-09-1/+22
| |
| * Swap out configMap for a hostPath for psuedo-persistenceGravatar Mark2021-01-09-2/+3
| | | | | | Co-authored-by: Joe Banks <[email protected]>
| * Add a Docker volume for the user baseGravatar MarkKoz2021-01-09-0/+13
| |
| * Install exposed packages separately during deploymentGravatar MarkKoz2021-01-09-438/+100
| |
| * Use a custom directory for exposed Python packagesGravatar MarkKoz2021-01-09-5/+7
|/ | | | | | | | | | | | | | | | | Isolate snekbox's dependencies from the packages available within the Python interpreter. Disable Python's default behaviour of site-dependent manipulations of sys.path. The custom directory looks like a user site to allow `pip install --user` to work with it. However, snekbox will see it as simply an additional search path for modules rather than as a user site. Disable isolated mode (-I) because it implies (-E), which ignores PYTHON* environment variables. This conflicts with the reliance on `PYTHONPATH`. Specify `PYTHONUSERBASE` in the Dockerfile to make installing packages to expose more intuitive for users. Otherwise, they'd have to remember to set this variable every time they need to install something.
* Merge pull request #86 from python-discord/precommit-pycharmGravatar Mark2021-01-02-1/+1
|\ | | | | Make flake8 properly run through pre-commit in PyCharm.
| * Changed Python language hook to system.Gravatar xithrius2021-01-02-1/+1
|/
* Merge PR #85 - Sentry releasesGravatar Mark2020-12-20-2/+32
|\
| * Fix getting Git SHA in workflowGravatar ks1292020-12-20-1/+1
| |
| * Add Sentry release to SDK initializationGravatar ks1292020-12-20-1/+3
| |
| * Create workflow for Sentry release creationGravatar ks1292020-12-20-0/+24
| |
| * Add Git SHA build argument to Docker build in workflowGravatar ks1292020-12-20-0/+2
| |
| * Inject Git SHA to environment from build argumentGravatar ks1292020-12-20-1/+3
|/
* Add @Akarys42 to the codeownersGravatar Matteo Bertucci2020-11-27-1/+1
|
* Add Mark to code ownersGravatar Mark2020-11-24-2/+2
|
* Update CODEOWNERSGravatar Dennis Pham2020-11-24-2/+2
|
* Update CODEOWNERSGravatar Joe Banks2020-11-24-1/+2
|
* Update CODEOWNERSGravatar Joe Banks2020-11-24-1/+1
|
* Merge pull request #81 - Limit STDOUT to prevent OOMGravatar Mark2020-11-20-6/+76
|\
| * Fix typo in _consume_stdout docstringGravatar Sebastiaan Zeeff2020-11-21-1/+1
| |
| * Move exit code conversion to main functionGravatar Sebastiaan Zeeff2020-11-21-15/+13
| |
| * Add test for stdout output truncationGravatar Sebastiaan Zeeff2020-11-21-2/+21
| | | | | | | | | | | | I've added a test that checks if output exceeding the limit is correctly truncated. To make the test more robust, I've defined a constant for the read chunk size.
| * Convert negative exit codes into standard formGravatar Sebastiaan Zeeff2020-11-20-9/+15
| | | | | | | | | | | | | | When you send a signal `N` to a subprocess using Popen, it will return `-N` as its exit code. As the rest of the code returns signal exit codes as `128 + N`, we convert those negative exit codes into the standard form used by the rest of the code.
| * Document output truncation in READMEGravatar Sebastiaan Zeeff2020-11-20-0/+1
| |
| * Use SIGKILL instead of SIGTERM to terminate NsJailGravatar Sebastiaan Zeeff2020-11-20-4/+4
| | | | | | | | | | | | This new behavior matches how other limiters terminate the subprocess, resulting in a more consistency in the front-end for the end users as well.
| * Increase number of characters in each read chunkGravatar Sebastiaan Zeeff2020-11-20-1/+1
| | | | | | | | | | | | I've increased the number of characters in each chunk we read from stdout to 10_000. This means we now read roughly 10 KB - 40 KB in each chunk.
| * Keep output that took us over the output limitGravatar Sebastiaan Zeeff2020-11-20-10/+6
| | | | | | | | | | | | | | Previously, the chunk of output that took us over the output limit was dismissed. As we've already consumed it and it's not going to have a very large size, we might as well include it in the final output we return.
| * Refactor STDOUT consumer to separate functionGravatar Sebastiaan Zeeff2020-11-20-22/+39
| | | | | | | | | | | | The function now returns a single, joined string instead of a list of a list of strings. That way, we don't have to join the list in two different code branches.
| * Limit STDOUT to prevent OOM events in containerGravatar Sebastiaan Zeeff2020-11-20-5/+38
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | Recently, we discovered that for some code inputs, snekbox would get into an OOM event on the container level, seemingly bypassing the memory restrictions laid on code execution by NSJail. After investigating the issue, we identified the culprit to be the STDOUT pipe we use to get output back from NSJail: As output is piped out of the jailed process, it will be gathered outside of the NSJail in the main container process instead. This meant that our initial attempts of limiting the allowed filesize within the NSJail failed, as the OOM happened outside of the jailed environment. To mitigate the issue, I've written a loop that consumes the STDOUT pipe in chunks of 100 characters. Once the size of the accrued output reaches a certain limit (currently set to 1 MB), we send a SIGTERM signal to NSJail to terminate itself. The output up to that point will be relayed back to the caller. A minimal code snippet to trigger the event and the mitigation: ```py while True: print(" ") ``` I've included a test for this vulnerability in `tests/test_nsjail.py`.
* Set maximum file size to 10Mb through rlimit_fsizeGravatar Joe Banks2020-11-19-0/+1
| | | | | This will set the maximum size of a created file to be 10Mb, a fairly generous amount. The reason for this is that when a huge stdout is buffered it does not get affected by the memory protections of nsjail and is sent to the host container, which has potential to cause an OOM.
* Add privileged securityContext to deploymentGravatar Joe Banks2020-11-19-0/+2
|
* Merge pull request #80 from python-discord/lemon/kubernetes_manifestGravatar Joe Banks2020-11-19-2/+37
|\
| * Add production_build == 'true' condition on deploy.Gravatar Leon Sandøy2020-11-19-0/+2
| |
| * Add Kubernetes auto-deploy to the CI build stage.Gravatar Leon Sandøy2020-11-19-0/+15
| |
| * Add a Kubernetes deployment manifest.Gravatar Leon Sandøy2020-11-19-2/+20
|/ | | | Simple stuff. Basically copy-paste from site.
* Merge pull request #79 from ↵Gravatar Leon Sandøy2020-11-19-412/+299
|\ | | | | | | | | python-discord/sebastiaan/backend/migrate-ci-to-github-actions Migrate to GitHub Actions and GitHub Container Registry
| * Fix Dockerfile paths & revert error in DockerfileGravatar Sebastiaan Zeeff2020-11-19-4/+4
| | | | | | | | | | I've fixed paths still pointing to the old Dockerfile location. I've also reverted an error that somehow got committed to the Dockerfile.
| * Remove redundant build commands for PipfileGravatar Sebastiaan Zeeff2020-11-18-17/+1
| | | | | | | | | | | | I've removed the redundant intermediate image build commands from the Pipfile. Since everything is now contained in one Dockerfile, we can simply build the final image in one go.
| * Use repository caching for master buildGravatar Sebastiaan Zeeff2020-11-18-39/+32
| | | | | | | | | | | | | | | | One problem that our master builds may have is that they retain more and more layers of old builds, as there is no easy way of purging them from the cache. As such master cache would not have benefits over using repository-based caching, I've removed persistent local caching for non-PR builds.
| * Rectify production branch and clean up formattingGravatar Sebastiaan Zeeff2020-11-18-14/+15
| |
| * Remove escape sequence and improve job nameGravatar Sebastiaan Zeeff2020-11-18-2/+2
| | | | | | | | | | | | I accidentally escaped a single quote in a run command; I've removed it now. I also changed the job name to `lint-test-build-push` to better reflect the contents of the job.
| * Remove CI dependency that is no longer in useGravatar Sebastiaan Zeeff2020-11-18-51/+46
| | | | | | | | | | | | Now we've migrated to GitHub Actions, we don't need have XML reports of our unit tests as we're no longer using the Azure test result application.
generated by cgit v1.2.3 (git 2.39.1) at 2025-10-19 16:11:17 +0000