2332:431ad52412bc | 23-Feb-2023 |
Andrew Clayton |
Set a safer umask(2) when running as a daemon.
When running as a daemon. unit currently sets umask(0), i.e no umask. This is resulting in various directories being created with a mode of 0777, e.g
Set a safer umask(2) when running as a daemon.
When running as a daemon. unit currently sets umask(0), i.e no umask. This is resulting in various directories being created with a mode of 0777, e.g
rwxrwxrwx
this is currently affecting cgroup and rootfs directories, which are being created with a mode of 0777, and when running as a daemon as there is no umask to restrict the permissions.
This also affects the language modules (the umask is inherited over fork(2)) whereby unless something explicitly sets a umask, files and directories will be created with full permissions, 0666 (rw-rw-rw-)/ 0777 (rwxrwxrwx) respectively.
This could be an unwitting security issue.
My original idea was to just remove the umask(0) call and thus inherit the umask from the executing shell/program.
However there was some concern about just inheriting whatever umask was in effect.
Alex suggested that rather than simply removing the umask(0) call we change it to a value of 022 (which is a common default), which will result in directories and files with permissions at most of 0755 (rwxr-xr-x) & 0644 (rw-r--r--).
If applications need some other umask set, they can (as they always have been able to) set their own umask(2).
Suggested-by: Alejandro Colomar <alx.manpages@gmail.com> Reviewed-by: Liam Crilly <liam@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2331:48798e038044 | 22-Feb-2023 |
Andrew Clayton |
Isolation: rootfs: Set the sticky bit on the tmp directory.
When using the 'rootfs' isolation option, by default a tmpfs filesystem is mounted on tmp/. Currently this is mounted with a mode of 0777,
Isolation: rootfs: Set the sticky bit on the tmp directory.
When using the 'rootfs' isolation option, by default a tmpfs filesystem is mounted on tmp/. Currently this is mounted with a mode of 0777, i.e
drwxrwxrwx. 3 root root 60 Feb 22 11:56 tmp
however this should really have the sticky bit[0] set (as is per-normal for such directories) to prevent users from having free reign on the files contained within.
What we really want is it mounted with a mode of 01777, i.e
drwxrwxrwt. 3 root root 60 Feb 22 11:57 tmp
[0]: To quote inode(7)
"The sticky bit (S_ISVTX) on a directory means that a file in that directory can be renamed or deleted only by the owner of the file, by the owner of the directory, and by a privileged process."
Reviewed-by: Liam Crilly <liam@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2326:f0a41e2f9f14 | 01-Dec-2022 |
Andrew Clayton |
Remove the nxt_getpid() alias.
Since the previous commit, nxt_getpid() is only ever aliased to getpid(2).
nxt_getpid() was only used once in the code, while there are multiple direct uses of getpid
Remove the nxt_getpid() alias.
Since the previous commit, nxt_getpid() is only ever aliased to getpid(2).
nxt_getpid() was only used once in the code, while there are multiple direct uses of getpid(2)
$ grep -r "getpid()" src/ src/nxt_unit.c: nxt_unit_pid = getpid(); src/nxt_process.c: nxt_pid = nxt_getpid(); src/nxt_process.c: nxt_pid = getpid(); src/nxt_lib.c: nxt_pid = getpid(); src/nxt_process.h:#define nxt_getpid() \ src/nxt_process.h:#define nxt_getpid() \ src/nxt_process.h: getpid()
Just remove it and convert the _single_ instance of nxt_getpid() to getpid(2).
Reviewed-by: Alejandro Colomar <alx@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2325:2f754a2ca783 | 19-Nov-2022 |
Andrew Clayton |
Isolation: Remove the syscall(SYS_getpid) wrapper.
When using SYS_clone we used the getpid kernel system call directly via syscall(SYS_getpid) to avoid issues with cached pids.
However since we are
Isolation: Remove the syscall(SYS_getpid) wrapper.
When using SYS_clone we used the getpid kernel system call directly via syscall(SYS_getpid) to avoid issues with cached pids.
However since we are now only using fork(2) (+ unshare(2) for namespaces) we no longer need to call the kernel getpid directly as the fork(2) will ensure the cached pid is invalidated.
Reviewed-by: Alejandro Colomar <alx@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2324:8621e6a00a69 | 19-Nov-2022 |
Andrew Clayton |
Isolation: Remove nxt_clone().
Since the previous commit, this is no longer used.
Reviewed-by: Alejandro Colomar <alx@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com> |
2323:f8b2c06cbd9a | 18-Nov-2022 |
Andrew Clayton |
Isolation: Switch to fork(2) & unshare(2) on Linux.
On GitHub, @razvanphp & @hbernaciak both reported issues running the APCu PHP module under Unit.
When using this module they were seeing errors l
Isolation: Switch to fork(2) & unshare(2) on Linux.
On GitHub, @razvanphp & @hbernaciak both reported issues running the APCu PHP module under Unit.
When using this module they were seeing errors like
'apcu_fetch(): Failed to acquire read lock'
However when running APCu under php-fpm, everything was fine.
The issue turned out to be due to our use of SYS_clone breaking the pthreads(7) API used by APCu. Even if we had been using glibc's clone(2) wrapper we would still have run into problems due to a known issue there.
Essentially the problem is when using clone, glibc doesn't update the TID cache, so the child ends up having the same TID as the parent and that is used in various parts of pthreads(7) such as in the various locking primitives, so when APCu was grabbing a lock it ended up using the TID of the main unit process (rather than that of the php application processes that was grabbing the lock).
So due to the above what was happening was when one of the application processes went to grab either a read or write lock, the lock was actually being attributed to the main unit process. If a process had acquired the write lock, then if a process tried to acquire a read or write lock then glibc would return EDEADLK due to detecting a deadlock situation due to thinking the process already held the write lock when in fact it didn't.
It seems the right way to do this is via fork(2) and unshare(2). We already use fork(2) on other platforms.
This requires a few tricks to keep the essence of the processes the same as before when using clone
1) We use the prctl(2) PR_SET_CHILD_SUBREAPER option (if its available, since Linux 3.4) to make the main unit process inherit prototype processes after a double fork(2), rather than them being reparented to 'init'.
This avoids needing to ^C twice to fully exit unit when running in the foreground. It's probably also better if they maintain their parent child relationship where possible.
2) We use a double fork(2) technique on the prototype processes to ensure they themselves end up in a new PID namespace as PID 1 (when CLONE_NEWPID is being used).
When using unshare(CLONE_NEWPID), the calling process is _not_ placed in the namespace (as discussed in pid_namespaces(7)). It only sets things up so that subsequent children are placed in a PID namespace.
Having the prototype processes as PID 1 in the new PID namespace is probably a good thing and matches the behaviour of clone(2). Also, some isolation tests break if the prototype process is not PID 1.
3) Due to the above double fork(2) the main unit process looses track of the prototype process ID, which it needs to know.
To solve this, we employ a simple pipe(2) between the main unit and prototype processes and pass the prototype grandchild PID from the parent of the second fork(2) before exiting. This needs to be done from the parent and not the grandchild, as the grandchild will see itself having a PID of 1 while the main process needs its externally visible PID.
Link: <https://www.php.net/manual/en/book.apcu.php> Link: <https://sourceware.org/bugzilla/show_bug.cgi?id=21793> Closes: <https://github.com/nginx/unit/issues/694> Reviewed-by: Alejandro Colomar <alx@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2321:b8d29a14676d | 18-Nov-2022 |
Andrew Clayton |
Isolation: Rename NXT_HAVE_CLONE -> NXT_HAVE_LINUX_NS.
Due to the need to replace our use of clone/__NR_clone on Linux with fork(2)/unshare(2) for enabling Linux namespaces(7) to keep the pthreads(7
Isolation: Rename NXT_HAVE_CLONE -> NXT_HAVE_LINUX_NS.
Due to the need to replace our use of clone/__NR_clone on Linux with fork(2)/unshare(2) for enabling Linux namespaces(7) to keep the pthreads(7) API working. Let's rename NXT_HAVE_CLONE to NXT_HAVE_LINUX_NS, i.e name it after the feature, not how it's implemented, then in future if we change how we do namespaces again we don't have to rename this.
Reviewed-by: Alejandro Colomar <alx@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2318:6f198d76ee62 | 30-Jan-2023 |
Zhidao HONG |
NJS: adding the missing vm destruction.
This commit fixed the njs memory leak happened in the config validation, updating and http requests. |
2317:f0ba91932216 | 07-Feb-2023 |
Andrew Clayton |
Python: ASGI: Don't log asyncio.get_running_loop() errors.
This adds a check to nxt_python_asgi_get_event_loop() on the event_loop_func name in the case that running that function fails, and if it's
Python: ASGI: Don't log asyncio.get_running_loop() errors.
This adds a check to nxt_python_asgi_get_event_loop() on the event_loop_func name in the case that running that function fails, and if it's get_running_loop() that failed we skip printing an error message as this is an often expected behaviour since the previous commit and we don't want users reporting erroneous bugs.
This check will always happen regardless of Python version while it really only applies to Python >= 3.7, there didn't seem much point adding complexity to the code for this case and in what will be an ever diminishing case of people running older Pythons.
Reviewed-by: Alejandro Colomar <alx@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2316:929b54de98c6 | 20-Jan-2023 |
Andrew Clayton |
Python: ASGI: Switch away from asyncio.get_event_loop().
Several users on GitHub reported issues with running Python ASGI apps on Unit with Python 3.11.1 (this would also effect Python 3.10.9) with
Python: ASGI: Switch away from asyncio.get_event_loop().
Several users on GitHub reported issues with running Python ASGI apps on Unit with Python 3.11.1 (this would also effect Python 3.10.9) with the following error from Unit
2023/01/15 22:43:22 [alert] 0#77128 [unit] Python failed to call 'asyncio.get_event_loop'
TL;DR
asyncio.get_event_loop() is currently broken due to the process of deprecating part or all of it.
First some history.
In Unit we had this commit
commit 8dcb0b9987033d0349a6ecf528014a9daa574787 Author: Max Romanov <max.romanov@nginx.com> Date: Thu Nov 5 00:04:59 2020 +0300
Python: request processing in multiple threads.
One of things this did was to create a new asyncio event loop in each thread using asyncio.new_event_loop().
It's perhaps worth noting that all these asyncio.* functions are Python functions that we call from the C code in Unit.
Then we had this commit
commit f27fbd9b4d2bdaddf1e7001d0d0bc5586ba04cd4 Author: Max Romanov <max.romanov@nginx.com> Date: Tue Jul 20 10:37:54 2021 +0300
Python: using default event_loop for main thread for ASGI.
This changed things so that Unit calls asyncio.get_event_loop() in the _main_ thread (but still calls asyncio.new_event_loop() in the other threads).
asyncio.get_event_loop() up until recently would either return an already running event loop or return a newly created one.
This was done for $reasons that the commit message and GitHub issue #560 hint at. But the intimation is that there can already be an event loop running from the application (I assume it's referring to the users application) at this point and if there is we should use it.
Now for the Python side of things.
On the main branch we had
commit 172c0f2752d8708b6dda7b42e6c5a3519420a4e8 Author: Serhiy Storchaka <storchaka@gmail.com> Date: Sun Apr 25 13:40:44 2021 +0300
bpo-39529: Deprecate creating new event loop in asyncio.get_event_loop() (GH-23554)
This commit began the deprecating of asyncio.get_event_loop().
commit fd38a2f0ec03b4eec5e3cfd41241d198b1ee555a Author: Serhiy Storchaka <storchaka@gmail.com> Date: Tue Dec 6 19:42:12 2022 +0200
gh-93453: No longer create an event loop in get_event_loop() (#98440)
This turned asyncio.get_event_loop() into a RuntimeError _if_ there isn't a current event loop.
commit e5bd5ad70d9e549eeb80aadb4f3ccb0f2f23266d Author: Serhiy Storchaka <storchaka@gmail.com> Date: Fri Jan 13 14:40:29 2023 +0200
gh-100160: Restore and deprecate implicit creation of an event loop (GH-100410)
This re-creates the event loop if there wasn't one and emits a deprecation warning.
After at least the last two commits Unit no longer works with the Python _main_ branch.
Meanwhile on the 3.11 branch we had
commit 3fae04b10e2655a20a3aadb5e0d63e87206d0c67 Author: Serhiy Storchaka <storchaka@gmail.com> Date: Tue Dec 6 17:15:44 2022 +0200
[3.11] gh-93453: Only emit deprecation warning in asyncio.get_event_loop when a new event loop is created (#99949)
which is what caused our breakage, though perhaps unintentionally as we get the following traceback
Traceback (most recent call last): File "/usr/lib64/python3.11/asyncio/events.py", line 676, in get_event_loop f = sys._getframe(1) ^^^^^^^^^^^^^^^^ ValueError: call stack is not deep enough 2023/01/18 02:46:10 [alert] 0#180279 [unit] Python failed to call 'asyncio.get_event_loop'
However, regardless, it is clear we need to stop using asyncio.get_event_loop().
One option is to switch to the higher level asyncio.run() API, however that is a rather large change.
This commit takes the simpler approach of using asyncio.get_running_loop() (which it seems get_event_loop() will eventually be an alias of) in the _main_ thread to return the currently running event loop, or if there is no current event loop, it will call asyncio.new_event_loop() to return a newly created event loop.
I believe this mimics the current behaviour. In my testing get_event_loop() seemed to always return a newly created loop, as when just calling get_running_loop() it would return NULL and we would fail out.
When running two processes each with 2 threads we would get the following loops with Python 3.11.0 and unpatched Unit
<_UnixSelectorEventLoop running=False closed=False debug=False> <_UnixSelectorEventLoop running=False closed=False debug=False> <_UnixSelectorEventLoop running=False closed=False debug=False> <_UnixSelectorEventLoop running=False closed=False debug=False>
and with Python 3.11.1 and a patched Unit we would get
<_UnixSelectorEventLoop running=False closed=False debug=False> <_UnixSelectorEventLoop running=False closed=False debug=False> <_UnixSelectorEventLoop running=False closed=False debug=False> <_UnixSelectorEventLoop running=False closed=False debug=False>
Tested-by: Rafał Safin <rafal.safin12@gmail.com> Reviewed-by: Alejandro Colomar <alx@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2315:624107a6485f | 20-Jan-2023 |
Andrew Clayton |
Python: ASGI: Factor out event loop creation to its own function.
This is a preparatory patch that factors out the asyncio event loop creation code from nxt_python_asgi_ctx_data_alloc() into its own
Python: ASGI: Factor out event loop creation to its own function.
This is a preparatory patch that factors out the asyncio event loop creation code from nxt_python_asgi_ctx_data_alloc() into its own function, to facilitate being called multiple times.
This a part of the work to move away from using the asyncio.get_event_loop() function due to it no longer creating event loops if there wasn't one running.
See the following commit for the gory details.
Reviewed-by: Alejandro Colomar <alx@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2314:bc5a90e2e6e8 | 14-Jul-2022 |
Alejandro Colomar |
Added default values for pathnames.
This allows one to simply run `./configure` and expect it to produce sane defaults for an install.
Previously, without specifying `--prefix=...`, `make install`
Added default values for pathnames.
This allows one to simply run `./configure` and expect it to produce sane defaults for an install.
Previously, without specifying `--prefix=...`, `make install` would simply fail, recommending to set `--prefix` or `DESTDIR`, but that recommendation was incomplete at best, since it didn't set many of the subdirs needed for a good organization.
Setting `DESTDIR` was even worse, since that shouldn't even affect an installation (it is required to be transparent to the installation).
/usr/local is the historic Unix standard path to use for installations from source made manually by the admin of the system. Some package managers (Homebrew, I'm looking specifically at you) have abused that path to install their things, but 1) it's not our fault that someone else incorrectly abuses that path (and they seem to be fixing it for newer archs; e.g., they started using /opt/homebrew for Apple Silicon), 2) there's no better path than /usr/local, 3) we still allow changing it for systems where this might not be the desired path (MacOS Intel with hombrew), and 4) it's _the standard_.
See a related conversation with Ingo (OpenBSD maintainer):
On 7/27/22 16:16, Ingo Schwarze wrote: > Hi Alejandro, [...] > > Alejandro Colomar wrote on Sun, Jul 24, 2022 at 07:07:18PM +0200: >> On 7/24/22 16:57, Ingo Schwarze wrote: >>> Alejandro Colomar wrote on Sun, Jul 24, 2022 at 01:20:46PM +0200: > >>>> /usr/local is for sysadmins to build from source; > >>> Doing that is *very* strongly discouraged on OpenBSD. > >> I guess that's why the directory was reused in the BSDs to install ports >> (probably ports were installed by the sysadmin there, and by extension, >> ports are now always installed there, but that's just a guess). > > Maybe. In any case, the practice of using /usr/local for packages > created from ports is significantly older than the recommendation > to refrain from using upstream "make install" outside the ports > framework. > > * The FreeBSD ports framework was started by Jordan Hubbard in 1993. > * The ports framework was ported from FreeBSD to OpenBSD > by Niklas Hallqvist in 1996. > * NetBSD pkgsrc was forked from FreeBSD ports by Alistair G. Crooks > and Hubert Feyrer in 1997. > > I failed to quickly find Jordan's original version, but rev. 1.1 > of /usr/ports/infrastructure/mk/bsd.port.mk in OpenBSD (dated Jun 3 > 22:47:10 1996 UTC) already said > > LOCALBASE ?= /usr/local > PREFIX ?= ${LOCALBASE} > [...] >> I had a discussion in NGINX Unit about it, and >> the decission for now has been: "support prefix=/usr/local for default >> manual installation through the Makefile, and let BSD users adjust to >> their preferred path". > > That's an *excellent* solution for the task, thanks for doing it > the right way. By setting PREFIX=/usr/local by default in the > upstream Makefile, you are minimizing the work for *BSD porters. > > The BSD ports frameworks will typically run the upstreak "make install" > with the variable DESTDIR set to a custom value, for example > > DESTDIR=/usr/ports/pobj/groff-1.23.0/fake-amd64 > > so if the upstream Makefile sets PREFIX=/usr/local , > that's perfect, everything gets installed to the right place > without an intervention by the person doing the porting. > > Of course, if the upstream Makefile would use some other PREFIX, > that would not be a huge obstacle. All we have to do in that case > is pass the option --prefix=/usr/local to the ./configure script, > or something equivalent if the software isn't using GNU configure. > >> We were concerned that we might get collisions >> with the BSD port also installing in /usr/local, but that's the least >> evil (and considering BSD users don't typically run `make install`, it's >> not so bad). > > It's not bad at all. It's perfect. > > Of course, if a user wants to install *without* the ports framework, > they have to provide their own --prefix. But that's not an issue > because it is easy to do, and installing without a port is discouraged > anyway.
===
Directory variables should never contain a trailing slash (I've learned that the hard way, where some things would break unexpectedly). Especially, make(1) is likely to have problems when things have double slashes or a trailing slash, since it treats filenames as text strings. I've removed the trailing slash from the prefix, and added it to the derivate variables just after the prefix. pkg-config(1) also expects directory variables to have no trailing slash.
===
I also removed the code that would set variables as depending on the prefix if they didn't start with a slash, because that is a rather non-obvious behavior, and things should not always depend on prefix, but other dirs such as $(runstatedir), so if we keep a similar behavior it would be very unreliable. Better keep variables intact if set, or use the default if unset.
===
Print the real defaults for ./configure --help, rather than the actual values.
===
I used a subdirectory under the standard /var/lib for NXT_STATE, instead of a homemade "state" dir that does the same thing.
===
Modified the Makefile to create some dirs that weren't being created, and also remove those that weren't being removed in uninstall, probably because someone forgot to add them.
===
Add new options for setting the new variables, and rename some to be consistent with the standard names. Keep the old ones at configuration time for compatibility, but mark them as deprecated. Don't keep the old ones at exec time.
===
A summary of the default config is:
Unit configuration summary:
bin directory: ............. "/usr/local/bin" sbin directory: ............ "/usr/local/sbin" lib directory: ............. "/usr/local/lib" include directory: ......... "/usr/local/include" man pages directory: ....... "/usr/local/share/man" modules directory: ......... "/usr/local/lib/unit/modules" state directory: ........... "/usr/local/var/lib/unit" tmp directory: ............. "/tmp"
pid file: .................. "/usr/local/var/run/unit/unit.pid" log file: .................. "/usr/local/var/log/unit/unit.log"
control API socket: ........ "unix:/usr/local/var/run/unit/control.unit.sock"
Link: <https://www.gnu.org/prep/standards/html_node/Directory-Variables.html> Link: <https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html> Reviewed-by: Artem Konev <a.konev@f5.com> Reviewed-by: Andrew Clayton <a.clayton@nginx.com> Tested-by: Andrew Clayton <a.clayton@nginx.com> Reviewed-by: Konstantin Pavlov <thresh@nginx.com> Signed-off-by: Alejandro Colomar <alx@nginx.com>
show more ...
|
2312:5abe32b6681f | 26-Jan-2023 |
Andrew Clayton |
PHP: Implement better error handling.
Previously the PHP module would produce one of four status codes
200 OK 301 Moved Permanently 500 Internal Server Error 503 Service Unavailable
200 fo
PHP: Implement better error handling.
Previously the PHP module would produce one of four status codes
200 OK 301 Moved Permanently 500 Internal Server Error 503 Service Unavailable
200 for successful requests, 301 for cases where the url was a directory without a trailing '/', 500 for bad PHP or non-existing PHP file and 503 for all other errors.
With this commit we now handle missing files and directories, returning 404 Not Found and files and directories that don't allow access, returning 403 Forbidden.
We do these checks in two places, when we check if we should do a directory redirect (bar -> bar/) and in the nxt_php_execute() function.
One snag with the latter is that the php_execute_script() function only returns success/failure (no reason). However while it took a zend_file_handle structure with the filename of the script to run, we can instead pass through an already opened file-pointer (FILE *) via that structure. So we can try opening the script ourselves and do the required checks before calling php_execute_script().
We also make use of the zend_stream_init_fp() function that initialises the zend_file_handle structure if it's available otherwise we use our own version. This is good because the zend_file_handle structure has changed over time and the zend_stream_init_fp() function should change with it.
Closes: <https://github.com/nginx/unit/issues/767> Reviewed-by: Alejandro Colomar <alx@nginx.com> Cc: Andrei Zeliankou <zelenkov@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2311:93cdf8cabdd5 | 25-Jan-2023 |
Andrew Clayton |
PHP: Simplify ctx->script_filename.start in nxt_php_execute().
Create a const char *filename variable to hold ctx->script_filename.start, which is a much more manageable name and will negate the nee
PHP: Simplify ctx->script_filename.start in nxt_php_execute().
Create a const char *filename variable to hold ctx->script_filename.start, which is a much more manageable name and will negate the need for any more casting in the following commit when we switch to using a FILE * instead of a filename in php_execute_script().
Reviewed-by: Alejandro Colomar <alx@nginx.com> Cc: Andrei Zeliankou <zelenkov@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2310:7792a1eda257 | 27-Jan-2023 |
Andrew Clayton |
PHP: Make use of zend_stream_init_filename().
Where possible make use of the zend_stream_init_filename() function introduced in PHP 7.4.
This is essentially a preparatory patch for switching to usi
PHP: Make use of zend_stream_init_filename().
Where possible make use of the zend_stream_init_filename() function introduced in PHP 7.4.
This is essentially a preparatory patch for switching to using an already opened file-pointer in nxt_php_execute(). While wrapping this new code in a PHP version check with a fallback to our own function is perhaps slightly overkill, it does reduce the diff of the commit that switches to a FILE *.
Reviewed-by: Alejandro Colomar <alx@nginx.com> Cc: Andrei Zeliankou <zelenkov@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2309:4e6dca345a28 | 27-Jan-2023 |
Alejandro Colomar |
PHP: Factored out code into a helper function.
We're going to use zend_stream_init_filename in a following commit. To reduce the diff of that change, move the current code that will be replaced, to
PHP: Factored out code into a helper function.
We're going to use zend_stream_init_filename in a following commit. To reduce the diff of that change, move the current code that will be replaced, to a function that has the same interface.
We use strlen(3) here to be able to use an interface without passing the length, but we will remove that call in a following code, so it has no performance issues.
Co-developed-by: Andrew Clayton <a.clayton@nginx.com> Signed-off-by: Alejandro Colomar <alx@nginx.com> Reviewed-by: Andrew Clayton <a.clayton@nginx.com> Cc: Andrei Zeliankou <zelenkov@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2304:ca6c1b6c7ed8 | 17-Jan-2023 |
Zhidao HONG |
NJS: added the keys API for the request objects.
This commit is to loop through the request objects headers, arguments, and cookies. |
2303:5ae42a44e2ab | 07-Nov-2022 |
Andrew Clayton |
PHP: Fix a potential problem parsing the path.
@dward on GitHub reported an issue with a URL like
http://foo.bar/test.php?blah=test.php/foo
where we would end up trying to run the script
test
PHP: Fix a potential problem parsing the path.
@dward on GitHub reported an issue with a URL like
http://foo.bar/test.php?blah=test.php/foo
where we would end up trying to run the script
test.php?blah=test.php
In the PHP module the format 'file.php/' is treated as a special case in nxt_php_dynamic_request() where we check the _path_ part of the url for the string '.php/'.
The problem is that the path actually also contains the query string, thus we were finding 'test.php/' in the above URL and treating that whole path as the script to run.
The fix is simple, replace the strstr(3) with a memmem(3), where we can limit the amount of path we use for the check.
The trick here and what is not obvious from the code is that while path.start points to the whole path including the query string, path.length only contains the length of the _path_ part.
NOTE: memmem(3) is a GNU extension and is neither specified by POSIX or ISO C, however it is available on a number of other systems, including: FreeBSD, OpenBSD, NetBSD, illumos, and macOS.
If it comes to it we can implement a simple alternative for systems which lack memmem(3).
This also adds a test case (provided by @dward) to cover this.
Closes: <https://github.com/nginx/unit/issues/781> Cc: Andrei Zeliankou <zelenkov@nginx.com> Reviewed-by: Alejandro Colomar <alx@nginx.com> Reviewed-by: Andrei Zeliankou <zelenkov@nginx.com> [test] Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2302:5f37386e94ee | 02-Dec-2022 |
Andrew Clayton |
Fix endianness detection in nxt_websocket_header_t.
The nxt_websocket_header_t structure defines the layout of a websocket frame header. As the websocket frame is mapped directly onto this structur
Fix endianness detection in nxt_websocket_header_t.
The nxt_websocket_header_t structure defines the layout of a websocket frame header. As the websocket frame is mapped directly onto this structure its layout needs to match how it's coming off the network.
The network being big endian means on big endian systems the structure layout can simply match that of the websocket frame header. On little endian systems we need to reverse the two bytes.
This was done via the BYTE_ORDER, BIG_ENDIAN and LITTLE_ENDIAN macros, however these are not universal, e.g they are _not_ defined on illumos (OpenSolaris / OpenIndiana) and so we get the following compiler errors
In file included from src/nxt_h1proto.c:12:0: src/nxt_websocket_header.h:25:13: error: duplicate member 'opcode' uint8_t opcode:4; ^~~~~~ src/nxt_websocket_header.h:26:13: error: duplicate member 'rsv3' uint8_t rsv3:1; ^~~~ src/nxt_websocket_header.h:27:13: error: duplicate member 'rsv2' uint8_t rsv2:1; ^~~~ src/nxt_websocket_header.h:28:13: error: duplicate member 'rsv1' uint8_t rsv1:1; ^~~~ src/nxt_websocket_header.h:29:13: error: duplicate member 'fin' uint8_t fin:1; ^~~ src/nxt_websocket_header.h:31:13: error: duplicate member 'payload_len' uint8_t payload_len:7; ^~~~~~~~~~~ src/nxt_websocket_header.h:32:13: error: duplicate member 'mask' uint8_t mask:1; ^~~~
This commit fixes that by using the new NXT_HAVE_{BIG,LITTLE}_ENDIAN macros introduced in the previous commit.
Closes: <https://github.com/nginx/unit/issues/297> Fixes: e501c74 ("Introducing websocket support in router and libunit.") Reviewed-by: Alejandro Colomar <alx@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2300:dc815c3c2244 | 05-Jan-2023 |
Andrew Clayton |
Python: Fix enabling of UTF-8 in some situations.
There was a couple of reports of Python applications failing due to the following type of error
File "/opt/netbox/netbox/netbox/configuration.py",
Python: Fix enabling of UTF-8 in some situations.
There was a couple of reports of Python applications failing due to the following type of error
File "/opt/netbox/netbox/netbox/configuration.py", line 25, in _import print(f"\U0001f9ec loaded config '{path}'") UnicodeEncodeError: 'ascii' codec can't encode character '\U0001f9ec' in position 0: ordinal not in range(128)
due to the use of Unicode text in the print() statement.
This only happened for python 3.8+ when using the "home" configuration option as this meant we were going through the new PyConfig configuration.
When using this new configuration method with the 'isolated' specific API (for embedded Python) UTF-8 is disabled by default, PyPreConfig->utf8_mode = 0.
To fix this we need to setup the Python pre config and enable utf-8 mode. However rather than enable utf-8 unconditionally we can set to it to -1 so that it will use the LC_CTYPE environment variable to determine whether to enable utf-8 mode or not. utf-8 mode will be enabled if LC_CTYPE is either: C, POSIX or some specific UTF-8 locale. This is the default utf8_mode setting when using the non-isolated PyPreConfig API.
Reported-by: Tobias Genannt <tobias.genannt@kappa-velorum.net> Tested-by: Tobias Genannt <tobias.genannt@kappa-velorum.net> Link: <https://peps.python.org/pep-0587/> Link: <https://docs.python.org/3/c-api/init_config.html#c.PyPreConfig.utf8_mode> Fixes: 491d0f70 ("Python: Added support for Python 3.11.") Closes: <https://github.com/nginx/unit/issues/817> Reviewed-by: Alejandro Colomar <alx@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2299:913f17ec2663 | 30-Dec-2022 |
Andrew Clayton |
Python: Do some cleanup in nxt_python3_init_config().
This is a preparatory patch for future work and cleans up the code a little in the Python 3.8+ variant of nxt_python3_init_config().
The main a
Python: Do some cleanup in nxt_python3_init_config().
This is a preparatory patch for future work and cleans up the code a little in the Python 3.8+ variant of nxt_python3_init_config().
The main advantage being we no longer have calls to PyConfig_Clear() in two different paths.
The variables have a little extra space in their declarations to allow for the next patch which introduces a variable with a longer type name, which will help reduce the size of the diff.
Reviewed-by: Alejandro Colomar <alx@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
Revision tags: 1.29.0-1, 1.29.0 |
|
2273:a13c26f62477 | 14-Nov-2021 |
OutOfFocus4 |
Python: Added "prefix" to configuration.
This patch gives users the option to set a `"prefix"` attribute for Python applications, either at the top level or for specific `"target"`s. If the attribut
Python: Added "prefix" to configuration.
This patch gives users the option to set a `"prefix"` attribute for Python applications, either at the top level or for specific `"target"`s. If the attribute is present, the value of `"prefix"` must be a string beginning with `"/"`. If the value of the `"prefix"` attribute is longer than 1 character and ends in `"/"`, the trailing `"/"` is stripped.
The purpose of the `"prefix"` attribute is to set the `SCRIPT_NAME` context value for WSGI applications and the `root_path` context value for ASGI applications, allowing applications to properly route requests regardless of the path that the server uses to expose the application.
The context value is only set if the request's URL path begins with the value of the `"prefix"` attribute. In all other cases, the `SCRIPT_NAME` or `root_path` values are not set. In addition, for WSGI applications, the value of `"prefix"` will be stripped from the beginning of the request's URL path before it is sent to the application.
Reviewed-by: Andrei Zeliankou <zelenkov@nginx.com> Reviewed-by: Artem Konev <artem.konev@nginx.com> Signed-off-by: Alejandro Colomar <alx@nginx.com>
show more ...
|
2272:ab94a5ec1978 | 22-Nov-2022 |
OutOfFocus4 |
Removed dead code.
Signed-off-by: Alejandro Colomar <alx@nginx.com> |
2266:3f66d6e064e2 | 06-Dec-2022 |
Andrew Clayton |
Configuration: made large_header_buffers a valid setting.
This is an extension to the previous commit, which made large_header_buffer_size a valid configuration setting.
This commit makes a related
Configuration: made large_header_buffers a valid setting.
This is an extension to the previous commit, which made large_header_buffer_size a valid configuration setting.
This commit makes a related value, large_header_buffers, a valid configuration setting.
While large_header_buffer_size effectively limits the maximum size of any single header (although unit will try to pack multiple headers into a buffer if they wholly fit).
large_header_buffers limits how many of these 'large' buffers are available. It makes sense to also allow this to be user set.
large_header_buffers is already set by the configuration system in nxt_router.c it just isn't set as a valid config option in nxt_conf_validation.c
With this change users can set this option in their config if required by the following
"settings": { "http": { "large_header_buffers": 8 } },
It retains its default value of 4 if this is not set.
NOTE: This is being released as undocumented and subject to change as it exposes internal workings of unit.
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|
2265:6c65127da239 | 04-May-2022 |
Andrew Clayton |
Configuration: made large_header_buffer_size a valid setting.
@JanMikes and @tagur87 on GitHub both reported issues with long URLs that were exceeding the 8192 byte large_header_buffer_size setting,
Configuration: made large_header_buffer_size a valid setting.
@JanMikes and @tagur87 on GitHub both reported issues with long URLs that were exceeding the 8192 byte large_header_buffer_size setting, which resulted in a HTTP 431 error (Request Header Fields Too Large).
This can be resolved in the code by updating the following line in src/nxt_router.c::nxt_router_conf_create()
skcf->large_header_buffer_size = 8192;
However, requiring users to modify unit and install custom versions is less than ideal. We could increase the value, but to what?
This commit takes the option of allowing the user to set this option in their config by making large_header_buffer_size a valid configuration setting.
large_header_buffer_size is already set by the configuration system in nxt_router.c it just isn't set as a valid config option in nxt_conf_validation.c
With this change users can set this option in their config if required by the following
"settings": { "http": { "large_header_buffer_size": 16384 } },
It retains its default value of 8192 bytes if this is not set.
With this commit, without the above setting or too low a value, with a long URL you get a 431 error. With the above setting set to a large enough value, the request is successful.
NOTE: This setting really determines the maximum size of any single header _value_. Also, unit will try and place multiple values into a buffer _if_ they fully fit.
NOTE: This is being released as undocumented and subject to change as it exposes internal workings of unit.
Closes: <https://github.com/nginx/unit/issues/521> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
show more ...
|