aboutsummaryrefslogtreecommitdiffstats
path: root/config (follow)
Commit message (Collapse)AuthorAgeLines
* Increase thread limit to 15Gravatar Joe Banks9 days-6/+6
| | | | | | | | | Changes to multiprocessing in Python 3.14 make it more likely that the 5 PID limit is hit even with non-complicated uses of multiprocessing. We have enough compute to allocate more PIDs and safely know this will not affect the operation of other services (we have since migrated our databases and heavier processing applications onto other hosts).
* Enable 3.13's colorized outputGravatar thurisatic2024-11-14-0/+1
|
* Update all references to /lang to /snekbinGravatar Chris Lovering2024-10-03-2/+2
|
* Allow setting a binary_path when calling /evalGravatar Chris Lovering2024-10-03-5/+0
|
* Solve issue #212Gravatar rahul50502a2024-06-29-1/+1
|
* Bump max memory to 70MiBGravatar Chris Lovering2024-01-24-1/+1
| | | | pandas now uses ~60MiB just to import, so the previous max was no longer enough.
* Refactor modules into subpackagesGravatar Mark2023-10-27-1/+1
|
* Merge branch 'main' into jb3/shared-memGravatar wookie1842023-08-30-26/+7
|\
| * Install multiple Python versions in imageGravatar Mark2023-08-19-26/+7
| | | | | | | | | | | | | | | | Separate snekbox's Python interpreter from the interpreter used by NsJail. This allows for the interpreters to be updated on different cadences and provides better isolation of packages. Each Python interpreter adds about 70 MB to the built image.
* | Enable a limited tmpfs for shared memoryGravatar Joe Banks2021-07-21-0/+8
|/
* Remove file ignore filter to use -B flag insteadGravatar Ionite2023-03-08-5/+1
|
* Add `config/.ignore` file, parsed to ignore file patterns for uploadGravatar Ionite2023-03-02-0/+4
| | | | Currently only includes some python bytecode files
* Move rlimit_fsize to cfgGravatar ionite342022-11-28-0/+1
|
* Readded lib64Gravatar ionite342022-11-22-0/+7
|
* Remove fsize in cfg to use defaultGravatar ionite342022-11-22-1/+0
|
* Move cfgs to file from process callGravatar ionite342022-11-22-8/+3
|
* Reverted rlimit_fsize in cfgGravatar ionite342022-11-22-2/+0
| | | | Removed rlimit fsize from cfg as it causes issues with protobuf parsing
* Corrected fsize limit to infGravatar Ionite2022-11-21-1/+1
|
* Set rlimit_fsize to unlimitedGravatar ionite342022-11-21-1/+1
|
* Increased PID limit to 6Gravatar Ionite2022-11-16-1/+1
|
* Merge branch 'main' into bytes-outputGravatar Ionite2022-11-16-3/+3
|\
| * Bump Python To 3.11Gravatar Hassan Abouelela2022-10-31-3/+3
| | | | | | | | | | | | | | Upgrade build image, configuration files, and documentation to python 3.11. Signed-off-by: Hassan Abouelela <[email protected]>
* | Increased nsjail file-size limitGravatar Ionite2022-11-16-0/+2
|/ | | | Increased limit from default 1MB to 128MB (per file)
* Fix WSGI app not being calledGravatar MarkKoz2022-06-01-1/+1
|
* Add __all__ to all modulesGravatar MarkKoz2022-05-30-1/+1
|
* Specify wsgi_app in Gunicorn configGravatar MarkKoz2022-05-30-0/+1
|
* Move logging code to separate utility modulesGravatar MarkKoz2022-05-30-1/+1
|
* Set the max swap size through NsJailGravatar MarkKoz2021-12-20-0/+1
|
* Bumps Python Version To 3.10Gravatar Hassan Abouelela2021-10-11-3/+3
| | | | Signed-off-by: Hassan Abouelela <[email protected]>
* chore: increase 3rd party thread limit environment variablesGravatar Joe Banks2021-07-17-5/+5
| | | | We define a few environment variables to stop third party libraries trying to default to spawning more processes, with the PID limit modification we can increase these values.
* feat: increased PID limitsGravatar Joe Banks2021-07-17-1/+1
| | | | Processes spawned in snekbox now have up to 5 PIDs available, each sharing the same memory limits and environment as the parent python process. As far as I could see in testing this does appear safe and processes behave as expected even when detatching from the parent or exceeding memory limits.
* Use PYTHONIOENCODING to enable utf-8 stdout for the nsjail pipe, and handle ↵Gravatar Bast2021-04-06-0/+1
| | | | | | | | | | the potential case where this is bypassable Since snekbox does not run with a tty, stdout is technically raw bytes, and thus incomplete surrogate pairs can be printed without the client application erroring, and instead fail within _consume_stdout when we attempt to decode it to a str. This commit sets the PYTHONIOENCODING environment variable to inform python to open the pipe in utf-8 mode. However, clever use of execl and os.unsetenv() can unset this environment variable, so we add a safety check to _consume_stdout to fail out of parsing output if it contains invalid unicode. This should only happen in deliberate cases, or significant bugs in python or a c library where output is printed to stdout ignoring the python stdout encoding.
* Remove cgroup parent constants from config fileGravatar Joe Banks2021-02-03-2/+0
|
* Use a custom directory for exposed Python packagesGravatar MarkKoz2021-01-09-1/+2
| | | | | | | | | | | | | | | | | Isolate snekbox's dependencies from the packages available within the Python interpreter. Disable Python's default behaviour of site-dependent manipulations of sys.path. The custom directory looks like a user site to allow `pip install --user` to work with it. However, snekbox will see it as simply an additional search path for modules rather than as a user site. Disable isolated mode (-I) because it implies (-E), which ignores PYTHON* environment variables. This conflicts with the reliance on `PYTHONPATH`. Specify `PYTHONUSERBASE` in the Dockerfile to make installing packages to expose more intuitive for users. Otherwise, they'd have to remember to set this variable every time they need to install something.
* Limit STDOUT to prevent OOM events in containerGravatar Sebastiaan Zeeff2020-11-20-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Recently, we discovered that for some code inputs, snekbox would get into an OOM event on the container level, seemingly bypassing the memory restrictions laid on code execution by NSJail. After investigating the issue, we identified the culprit to be the STDOUT pipe we use to get output back from NSJail: As output is piped out of the jailed process, it will be gathered outside of the NSJail in the main container process instead. This meant that our initial attempts of limiting the allowed filesize within the NSJail failed, as the OOM happened outside of the jailed environment. To mitigate the issue, I've written a loop that consumes the STDOUT pipe in chunks of 100 characters. Once the size of the accrued output reaches a certain limit (currently set to 1 MB), we send a SIGTERM signal to NSJail to terminate itself. The output up to that point will be relayed back to the caller. A minimal code snippet to trigger the event and the mitigation: ```py while True: print(" ") ``` I've included a test for this vulnerability in `tests/test_nsjail.py`.
* Set maximum file size to 10Mb through rlimit_fsizeGravatar Joe Banks2020-11-19-0/+1
| | | | | This will set the maximum size of a created file to be 10Mb, a fairly generous amount. The reason for this is that when a huge stdout is buffered it does not get affected by the memory protections of nsjail and is sent to the host container, which has potential to cause an OOM.
* Update Python binary mount path in nsjail configGravatar MarkKoz2020-11-14-2/+2
|
* Use a config file for gunicornGravatar MarkKoz2020-03-23-0/+5
| | | | | This will make it easy to maintain a consistent config without relying on invocation via pipenv.
* Move snekbox.cfg to a config directoryGravatar MarkKoz2020-03-23-0/+118
There will be more config files to come so it's cleaner to have them together than littering the root directory with more files.
generated by cgit v1.2.3 (git 2.39.1) at 2025-10-19 07:44:54 +0000