aboutsummaryrefslogtreecommitdiffstats
path: root/scripts (follow)
Commit message (Collapse)AuthorAgeLines
* Use the py_verison variable rather than reusing argv againGravatar Chris Lovering2024-10-12-2/+2
|
* Ensure Python versions ending in t are installed to a dir ending in tGravatar Chris Lovering2024-10-07-3/+11
| | | | The t denotes that it is a free threading version of Python, so could potentially be installed along side a non-free-threaded version of Python
* Update all references to /lang to /snekbinGravatar Chris Lovering2024-10-03-5/+5
|
* Also split on hyphens in build python scriptGravatar Chris Lovering2024-10-03-2/+2
| | | | This is needed as dev builds such as 3.13-dev use the suffix -dev, rather than a patch version.
* Use xargs over find -exec to ensure exit code is returned properlyGravatar Chris Lovering2024-04-07-2/+2
| | | | | Running this script in it's previous form (via `docker compose run`) always returned an exit code of 0. This is due to `find` always returning a 0 exit code, unless an error occurred while traversing the directories.
* Make CI and deployment.yaml share the same dependency install script (#190)Gravatar ChrisJL2023-08-31-0/+5
| | | | | | | | | * Make CI and deployment.yaml share the same dependency install script * squashme: add set -euo pipefail as sane defaults This also requires running under bash rather than sh * squashme: add make target for installing eval deps
* Install multiple Python versions in imageGravatar Mark2023-08-19-0/+22
| | | | | | | | Separate snekbox's Python interpreter from the interpreter used by NsJail. This allows for the interpreters to be updated on different cadences and provides better isolation of packages. Each Python interpreter adds about 70 MB to the built image.
* Fix commit count part of version being off by 1Gravatar MarkKoz2022-06-01-1/+2
|
* Docker: install package in image and use version to tag itGravatar MarkKoz2022-05-31-0/+4
|
* Automatically determine the package versionGravatar MarkKoz2022-05-31-0/+34
| | | | | | Use the HEAD commit's date as the package's version. Append the number of commits made on the same date as HEAD to ensure multiple releases on the same date still have unique versions.
* Rename Generated ProtoBuf ConfigGravatar Hassan Abouelela2021-12-20-2/+1
| | | | | | | Remove the rename step from the protobuf generation script to get around a bug causing failures in the test suite. Signed-off-by: Hassan Abouelela <[email protected]>
* Replace protoc shell script with a Python oneGravatar MarkKoz2021-02-04-14/+62
|
* Replace dev.sh with Docker ComposeGravatar MarkKoz2021-02-04-65/+0
| | | | | | | | | | Managing development containers through Docker Compose is convenient. However, it isn't quite flexible enough to facilitate both development and normal use. It's not really worth accommodating the latter since the container gets pushed to a registry and that's the intended way to run the service. Anyone that is checking out the repository and therefore has access to the compose file is likely a developer, not a user.
* Replace nsjpy alias with a Python scriptGravatar MarkKoz2021-01-10-23/+1
| | | | | | | | | The Python script uses the same underlying code Falcon uses to invoke nsjail. It allows for the omission of redundant shell code that set up cgroups and nsjail args. This is also a step towards removing dependence on shell scripts and thus resolving #73.
* Compile the NsJail config protobuf into Python codeGravatar MarkKoz2021-01-10-0/+14
| | | | Include a helper shell script for compilation.
* Add more pre-commit hooksGravatar MarkKoz2021-01-09-0/+1
| | | | | | | | | | | | | | | Hooks added: * check-merge-conflict - checks for files with merge conflict strings * check-toml - attempts to load all toml files to verify syntax * check-yaml - attempts to load all yaml files to verify syntax * end-of-file-fixer - ensures files end in a newline and only a newline * mixed-line-ending - replaces mixed line endings with LF * trailing-whitespace - trims trailing whitespace * python-check-blanket-noqa - enforces that noqa annotations always occur with specific codes See: python-discord/organisation#138
* Use a custom directory for exposed Python packagesGravatar MarkKoz2021-01-09-1/+1
| | | | | | | | | | | | | | | | | Isolate snekbox's dependencies from the packages available within the Python interpreter. Disable Python's default behaviour of site-dependent manipulations of sys.path. The custom directory looks like a user site to allow `pip install --user` to work with it. However, snekbox will see it as simply an additional search path for modules rather than as a user site. Disable isolated mode (-I) because it implies (-E), which ignores PYTHON* environment variables. This conflicts with the reliance on `PYTHONPATH`. Specify `PYTHONUSERBASE` in the Dockerfile to make installing packages to expose more intuitive for users. Otherwise, they'd have to remember to set this variable every time they need to install something.
* Fix Dockerfile paths & revert error in DockerfileGravatar Sebastiaan Zeeff2020-11-19-1/+1
| | | | | I've fixed paths still pointing to the old Dockerfile location. I've also reverted an error that somehow got committed to the Dockerfile.
* Migrate build pipeline to Github ActionsGravatar Sebastiaan Zeeff2020-11-18-131/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I've migrated the build pipeline to GitHub Actions and changed the container registry to GitHub Container Registry. In the process, I've made some changes to our docker setup and caching: - We are now using a single multi-stage Dockerfile Instead of three separate dockerfiles, we are now using a single multi-stage Dockerfile that can be used to build the three images we want using build targets. In part, this is because we're now using the docker buildx build action currently recommended by docker. This new engine runs in a sandboxed mode, meaning that while it can export built images to `docker` running in the host, it cannot import local images from it to base builds on. - Docker builds are now cached within GitHub Actions The builds are now cached using the GitHub Actions cache of the build cache directory. The cache keys try to match a cache generated by a build that matches the current build as closely as possible. In case of a cache miss, we fall back to caching from the latest image pushed to the container repository. - The `base` and `venv` images now have an inline cache manifest In order to fall back intelligently to caching from the repository, the final build and push action for the `base` and `venv` images includes an "inline" cache manifest. This means that the build process can inspect, without pulling, if it makes sense to pull layers to speed up the build. The other options, pushing a cache manifest separately (not inline), is currently not supported by GHCR. The custom caching script has been removed. - Linting errors are now added as GitHub Actions annotations Just like for some of our other pipelines, linting now generates annotations if linting errors are observed. - Coverage is pushed to coveralls.io A coverage summary is now pushed to coveralls.io. Each CI run will get a unique job that's linked in the CI output. If the run is attached to a PR, coveralls.io will automatically add a check link with the coverage result to the PR as well. - The README.md, Pipfile, docker-compose, and scripts have been updated As we now need to pull from and link to the GHCR, I've updated the other files to reflect these changes, including Pipfile run commands. I've also changed the CI badge and added a coveralls.io badge.
* Move snekbox.cfg to a config directoryGravatar MarkKoz2020-03-23-1/+1
| | | | | There will be more config files to come so it's cleaner to have them together than littering the root directory with more files.
* Change CI and dev.sh to not use host network mode.Gravatar scragly2020-02-29-1/+0
|
* CI: fix can_pull causing script to exit with code 1Gravatar MarkKoz2020-02-23-2/+2
| | | | | | | | Error handling is performed inside can_pull so the callers of the function don't always check its exit code. Because set -e present, if can_pull returns 1, bash would consider that function a failed call and thus exit the entire script with code 1. That, in turn, would cause the CI job to fail.
* CI: log when the build cache is usedGravatar MarkKoz2020-02-22-0/+1
|
* CI: fix unbound variable error when checking cacheGravatar MarkKoz2020-02-22-1/+1
| | | | The array shouldn't be expanded when testing with -v.
* CI: cache the response from Azure APIGravatar MarkKoz2020-02-22-0/+11
| | | | | | The script may need to use the master commit several times. The easiest way to implement the cache was to just cache the response rather than the commit hash.
* CI: check if venv image can be pulledGravatar MarkKoz2020-02-21-0/+6
|
* CI: create a function to check if an image can be pulledGravatar MarkKoz2020-02-21-13/+26
|
* CI: don't use output variablesGravatar MarkKoz2020-02-21-3/+3
| | | | | | It was initially used to enable access to variables across jobs, but the jobs will eventually be consolidated into one so output variables will no longer be needed.
* CI: diff base against master only if venv changedGravatar MarkKoz2020-02-21-16/+14
|
* CI: update agent to Ubuntu 18.04Gravatar MarkKoz2020-02-21-2/+1
| | | | * Use inherit_errexit in check_dockerfiles.sh
* CI: add more logging in check_dockerfiles.shGravatar MarkKoz2020-02-21-0/+9
|
* CI: always check if the base image can be pulled from Docker HubGravatar MarkKoz2020-02-21-2/+6
|
* CI: install Pipfile into system interpreterGravatar MarkKoz2020-02-21-1/+1
| | | | | | | | | | | A virtual environment is redundant in the context of deployment. It just increases the size and build time of the image. * Replace venv with system interpreter * Mount Python binaries in /usr/local/bin in NsJail * Fix #61: Python symlink in venv not resolving * Re-lock Pipfile because it wasn't up to date according to pipenv install --deploy
* Fix #56: stdout and stderr outputs in wrong orderGravatar MarkKoz2020-01-11-1/+1
|
* Mount only what's needed in the chroot jailGravatar MarkKoz2019-12-28-17/+1
| | | | | | | | | | | | devfs and sysfs were problematic since they were being mounted as tmpfs, which is r/w. For example, the Python process could write to cgroups. Now, only what is needed to run Python gets mounted. This boils down to the venv itself and some shared libraries Python needs. * Use a config file for NsJail instead of command-line options * Map 65534 (nobody) user & group inside the user namespace to 65534 outside the namespace rather than mapping to current uid/guid (which was 0 AKA root)
* Disable shared memory in Docker containerGravatar MarkKoz2019-12-28-0/+1
|
* Limit numpy to using 1 threadGravatar MarkKoz2019-12-16-0/+5
|
* Use bash instead of ashGravatar MarkKoz2019-12-14-5/+5
| | | | Forgot to do this after switching to Debian.
* Merge pull request #33 from python-discord/ciGravatar Johannes Christ2019-08-06-4/+91
|\ | | | | CI Improvements
| * Add comments to Azure Pipelines YAMLGravatar MarkKoz2019-07-30-4/+5
| | | | | | | | | | * Replace some shorthand Docker command options with their full names for clarity
| * CI: ensure count of builds returned by the API is > 0Gravatar MarkKoz2019-06-30-1/+2
| |
| * CI: refactor script & pull base when possibleGravatar MarkKoz2019-06-30-22/+76
| | | | | | | | | | | | | | | | * Move script's execution to the test job * Use output variables * Use jq instead of regex for parsing JSON responses from API * Wrap to 80 columns * Make more robust by checking for command success
| * CI: move check shell script to a separate fileGravatar MarkKoz2019-06-30-0/+31
| |
* | Use IDs for user and group in nsjpy aliasGravatar MarkKoz2019-08-04-2/+2
|/ | | | Reflects the changes in 7a7eca52019bf21d21cdffcf03cd9c5eacd8363b
* Disable memory swapping and add a memory limit testGravatar MarkKoz2019-06-26-1/+8
| | | | | | | | | If memory swapping was enabled locally, the memory test would fail. Explicitly disabling swapping also removes reliance on the assumption that it'll be disabled in production. * Add a constant for the maximum memory * Simplify the timeout test; it'd otherwise first run out of memory now
* Fix coverage not finding sourcesGravatar MarkKoz2019-06-22-8/+17
| | | | | | * Mount volume to the same path as the source directory on the host * Keep the container up in the background so it doesn't have to be restarted or the ownership fix
* Fix ownership of coverage fileGravatar MarkKoz2019-06-22-1/+10
| | | | | | When coverage runs in a container, it is ran under root so the resulting coverage file is owned by root. chown is used to change ownership to be the same as the folder it is in.
* Create shell script for building a dev image and running a shellGravatar MarkKoz2019-06-22-0/+70
| | | | * Put scripts in a new scripts folder
* Add Azure CI. (#16)Gravatar scragly2019-03-29-18/+0
| | | | | This PR is to add CI settings to master and to test the PR CI pipeline.
* add deploy scriptGravatar Christopher Baklid2018-07-29-0/+18
|
generated by cgit v1.2.3 (git 2.39.1) at 2025-10-19 05:15:48 +0000