1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
|
[](https://dev.azure.com/python-discord/Python%20Discord/_build/latest?definitionId=13&branchName=master)
# snekbox
Python sandbox runners for executing code in isolation aka snekbox.
A client sends Python code to a snekbox, the snekbox executes the code, and finally the results of the execution are returned to the client.
```
+-------------+ +-----------+
input -> | |---------->| | >----------+
| HTTP POST | | SNEKBOX | execution |
result <- | |<----------| | <----------+
+-------------+ +-----------+
^ ^
| |- Executes python code
| |- Returns result
| +-----------------------
|
|- HTTP POST Endpoint receives request and returns result
+---------------------------------------------------------
```
The code is executed in a Python process that is launched through [NsJail](https://github.com/google/nsjail), which is responsible for sandboxing the Python process. NsJail is configured as follows:
* Root directory is mounted as read-only
* Time limit of 2 seconds
* Maximum of 1 PID
* Maximum memory of 52428800 bytes
* Loopback interface is down
* procfs is disabled
The Python process is configured as follows:
* Version 3.7.4
* Isolated mode
* Neither the script's directory nor the user's site packages are in `sys.path`
* All `PYTHON*` environment variables are ignored
## HTTP REST API
Communication with snekbox is done over a HTTP REST API. The framework for the HTTP REST API is [Falcon](https://falconframework.org/) and the WSGI being used is [Gunicorn](https://gunicorn.org/). By default, the server is hosted on `0.0.0.0:8060` with two workers.
See [`snekapi.py`](snekbox/api/snekapi.py) and [`resources`](snekbox/api/resources) for API documentation.
## Development Environment
### Initial Setup
A Python 3.7 interpreter and the [pipenv](https://docs.pipenv.org/en/latest/) package are required. Once those requirements are satisfied, install the project's dependencies:
```
pipenv --sync
```
Follow that up with setting up the pre-commit hook:
```
pipenv run precommit
```
Now Flake8 will run and lint staged changes whenever an attempt to commit the changes is made. Flake8 can still be invoked manually:
```
pipenv run lint
```
### Running snekbox
The Docker images can be built with:
```
pipenv run buildbase
pipenv run buildvenv
pipenv run build
```
Use Docker Compose to start snekbox:
```
docker-compose up
```
### Running Tests
Tests are run through coverage.py using unittest. Before tests can run, the dev venv Docker image has to be built:
```
pipenv run builddev
```
Alternatively, the following command will build the image and then run the tests:
```
pipenv run testb
```
If the image doesn't need to be built, the tests can be run with:
```
pipenv run test
```
### Coverage
To see a coverage report, run
```
pipenv run report
```
Alternatively, a report can be generated as HTML:
```
pipenv run coverage html
```
The HTML will output to `./htmlcov/` by default
### The `devsh` Helper Script
This script starts an `ash` shell inside the venv Docker container and attaches to it. Unlike the production image, the venv image that is built by this script contains dev dependencies too. The project directory is mounted inside the container so any filesystem changes made inside the container affect the actual local project.
#### Usage
```
pipenv run devsh [--build [--clean]] [ash_args ...]
```
* `--build` Build the venv Docker image
* `--clean` Clean up dangling Docker images (only works if `--build` precedes it)
* `ash_args` Arguments to pass to `/bin/ash` (for example `-c "echo hello"`). An interactive shell is launched if no arguments are given
#### Invoking NsJail
A shell alias named `nsjpy` is included and is basically `nsjail python -c <args>` but NsJail is configured as it would be if snekbox invoked it (such as the time and memory limits). It provides an easy way to run Python code inside NsJail without the need to run snekbox with its webserver and send HTTP requests. Example usage:
```bash
nsjpy "print('hello world!')"
```
The alias can be found in `./scripts/.profile`, which is automatically added when the shell is launched in the container.
|
