diff options
| -rw-r--r-- | .github/workflows/lint-test-build-push.yaml | 3 | ||||
| -rw-r--r-- | Dockerfile | 29 | ||||
| -rw-r--r-- | README.md | 17 | ||||
| -rw-r--r-- | deployment.yaml | 1 |
4 files changed, 28 insertions, 22 deletions
diff --git a/.github/workflows/lint-test-build-push.yaml b/.github/workflows/lint-test-build-push.yaml index 671e576..d77ead1 100644 --- a/.github/workflows/lint-test-build-push.yaml +++ b/.github/workflows/lint-test-build-push.yaml @@ -107,11 +107,12 @@ jobs: docker exec snekbox_test /bin/bash -c 'apt-get -y update && apt-get install -y git=1:2.20.*' + # pre-commit's venv doesn't work with user installs. # Skip the flake8 hook because the following step will run it. - name: Run pre-commit hooks run: >- docker exec snekbox_test /bin/bash -c - 'SKIP=flake8 pre-commit run --all-files' + 'PIP_USER=0 SKIP=flake8 pre-commit run --all-files' # This runs `flake8` in the container and asks `flake8` to output # linting errors in the format of the command for registering workflow @@ -19,8 +19,17 @@ RUN git clone \ WORKDIR /nsjail RUN make +# ------------------------------------------------------------------------------ FROM python:3.9-slim-buster as base -ENV PIP_NO_CACHE_DIR=false + +# Everything will be a user install to allow snekbox's dependencies to be kept +# separate from the packages exposed during eval. +ENV PATH=/root/.local/bin:$PATH \ + PIP_NO_CACHE_DIR=false \ + PIP_USER=1 \ + PIPENV_DONT_USE_PYENV=1 \ + PIPENV_HIDE_EMOJIS=1 \ + PIPENV_NOSPIN=1 RUN apt-get -y update \ && apt-get install -y \ @@ -28,34 +37,26 @@ RUN apt-get -y update \ libnl-route-3-200=3.4.* \ libprotobuf17=3.6.* \ && rm -rf /var/lib/apt/lists/* -RUN pip install pipenv==2020.11.4 +RUN pip install pipenv==2020.11.15 COPY --from=builder /nsjail/nsjail /usr/sbin/ RUN chmod +x /usr/sbin/nsjail +# ------------------------------------------------------------------------------ FROM base as venv ARG DEV -ENV PIP_NO_CACHE_DIR=false \ - PIPENV_DONT_USE_PYENV=1 \ - PIPENV_HIDE_EMOJIS=1 \ - PIPENV_NOSPIN=1 \ - PYTHONUSERBASE=/snekbox/user_base - COPY Pipfile Pipfile.lock /snekbox/ WORKDIR /snekbox -RUN if [ -n "${DEV}" ]; \ - then \ - pipenv install --deploy --system --dev; \ - else \ - pipenv install --deploy --system; \ - fi +# Install to the default user site since PIP_USER is set. +RUN pipenv install --deploy --system ${DEV:+--dev} # At the end to avoid re-installing dependencies when only a config changes. # It's in the venv image because the final image is not used during development. COPY config/ /snekbox/config +# ------------------------------------------------------------------------------ FROM venv ENTRYPOINT ["gunicorn"] @@ -1,4 +1,7 @@ -[![Build Status][1]][2] [![Coverage Status][3]][4] +[![Discord][5]][6] +[![Build Status][1]][2] +[![Coverage Status][3]][4] +[](LICENSE) # snekbox @@ -50,19 +53,17 @@ By default, the Python interpreter has no access to any packages besides the standard library. Even snekbox's own dependencies like Falcon and Gunicorn are not exposed. -To expose third-party Python packages during evaluation, install them to the user site: +To expose third-party Python packages during evaluation, install them to a custom user site: ```sh -docker exec snekbox /bin/sh -c 'pip install --ignore-installed --user numpy' +docker exec snekbox /bin/sh -c 'PYTHONUSERBASE=/snekbox/user_base pip install numpy' ``` In the above command, `snekbox` is the name of the running container. The name may be different and can be checked with `docker ps`. -It's important to use `--user` to install them to the user site, whose base is located at `/snekbox/user_base` within the Docker container. To persist the installed packages, a volume for the directory can be created with Docker. For an example, see [`docker-compose.yml`]. +The packages will be installed to the user site within `/snekbox/user_base`. To persist the installed packages, a volume for the directory can be created with Docker. For an example, see [`docker-compose.yml`]. -`--ignore-installed` is only necessary if installing a package that happens to -be a dependency of snekbox. Normally, pip would reject the installation because -it doesn't make a distinction here between the global and user sites. +If `pip`, `setuptools`, or `wheel` are dependencies or need to be exposed, then use the `--ignore-installed` option with pip. However, note that this will also re-install packages present in the custom user site, effectively making caching it futile. Current limitations of pip don't allow it to ignore packages extant outside the installation destination. ## Development Environment @@ -165,6 +166,8 @@ With this command, NsJail uses the same configuration normally used through the [2]: https://github.com/python-discord/snekbox/actions?query=workflow%3A%22Lint%2C+Test%2C+Build%2C+Push%22+branch%3Amaster [3]: https://coveralls.io/repos/github/python-discord/snekbox/badge.svg?branch=master [4]: https://coveralls.io/github/python-discord/snekbox?branch=master +[5]: https://raw.githubusercontent.com/python-discord/branding/master/logos/badge/badge_github.svg +[6]: https://discord.gg/python [`snekbox.cfg`]: config/snekbox.cfg [`snekapi.py`]: snekbox/api/snekapi.py [`resources`]: snekbox/api/resources diff --git a/deployment.yaml b/deployment.yaml index 4545c3f..0b294d2 100644 --- a/deployment.yaml +++ b/deployment.yaml @@ -30,6 +30,7 @@ spec: - "/bin/sh" - "-c" - >- + PYTHONUSERBASE=/snekbox/user_base pip install --user arrow~=0.17 attrs~=20.3 |