/unit/tools/ |
H A D | unitc | diff 2475:b4f09ab4b9f6 Wed May 24 15:27:00 UTC 2023 Andrei Zeliankou <zelenkov@nginx.com> Updated copyright notice.
|
/unit/auto/modules/ |
H A D | conf | diff 2517:161f3197b5b6 Tue Aug 08 22:24:00 UTC 2023 Andrew Clayton <a.clayton@nginx.com> Wasm: Wire the Wasm language module up to the build system.
This allows to configure the Wasm module, e.g
./configure wasm --include-path=/path/to/wasmtime-v11.0.0-x86_64-linux-c-api/include --lib-path=/path/to/wasmtime-v11.0.0-x86_64-linux-c-api/lib --rpath
--rpath as above says to set the rpath to the value of --lib-path. You can alternatively specify a directory to use as the rpath. Or simply omit the option to not have an rpath set.
This is mostly useful for during development where you may not have the Wasmtime stuff installed to system directories or you want to test with newer/different versions.
See ./configure wasm --help for a full list of options.
Reviewed-by: Alejandro Colomar <alx@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
/unit/docs/ |
H A D | changes.xml | diff 2489:b37326158d10 Wed May 24 16:27:00 UTC 2023 Zhidao HONG <z.hong@f5.com> HTTP: fixed variable caching.
When a variable is accessed in the Unit configuration, the value is cached. This was useful prior to the URI rewrite feature, but now that the URI (more precisely, the request target) can be rewritten, the contents of the variable $uri (which contains the path part of the request target, and is decoded) should not be cached anymore, or at least the cached value should be invalidated after a URI rewrite.
Example:
{ "rewrite": "/prefix$uri", "share": "$uri" }
For a request line like GET /foo?bar=baz HTTP/1.1\r\n, the expected file served in the response would be /prefix/foo, but due to the caching issue, Unit currently serves /foo. diff 2474:00f56c19efeb Wed May 24 15:24:00 UTC 2023 Andrei Zeliankou <zelenkov@nginx.com> Version bump. diff 2474:00f56c19efeb Wed May 24 15:24:00 UTC 2023 Andrei Zeliankou <zelenkov@nginx.com> Version bump. diff 2314:bc5a90e2e6e8 Thu Jul 14 11:25:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Added default values for pathnames.
This allows one to simply run `./configure` and expect it to produce sane defaults for an install.
Previously, without specifying `--prefix=...`, `make install` would simply fail, recommending to set `--prefix` or `DESTDIR`, but that recommendation was incomplete at best, since it didn't set many of the subdirs needed for a good organization.
Setting `DESTDIR` was even worse, since that shouldn't even affect an installation (it is required to be transparent to the installation).
/usr/local is the historic Unix standard path to use for installations from source made manually by the admin of the system. Some package managers (Homebrew, I'm looking specifically at you) have abused that path to install their things, but 1) it's not our fault that someone else incorrectly abuses that path (and they seem to be fixing it for newer archs; e.g., they started using /opt/homebrew for Apple Silicon), 2) there's no better path than /usr/local, 3) we still allow changing it for systems where this might not be the desired path (MacOS Intel with hombrew), and 4) it's _the standard_.
See a related conversation with Ingo (OpenBSD maintainer):
On 7/27/22 16:16, Ingo Schwarze wrote: > Hi Alejandro, [...] > > Alejandro Colomar wrote on Sun, Jul 24, 2022 at 07:07:18PM +0200: >> On 7/24/22 16:57, Ingo Schwarze wrote: >>> Alejandro Colomar wrote on Sun, Jul 24, 2022 at 01:20:46PM +0200: > >>>> /usr/local is for sysadmins to build from source; > >>> Doing that is *very* strongly discouraged on OpenBSD. > >> I guess that's why the directory was reused in the BSDs to install ports >> (probably ports were installed by the sysadmin there, and by extension, >> ports are now always installed there, but that's just a guess). > > Maybe. In any case, the practice of using /usr/local for packages > created from ports is significantly older than the recommendation > to refrain from using upstream "make install" outside the ports > framework. > > * The FreeBSD ports framework was started by Jordan Hubbard in 1993. > * The ports framework was ported from FreeBSD to OpenBSD > by Niklas Hallqvist in 1996. > * NetBSD pkgsrc was forked from FreeBSD ports by Alistair G. Crooks > and Hubert Feyrer in 1997. > > I failed to quickly find Jordan's original version, but rev. 1.1 > of /usr/ports/infrastructure/mk/bsd.port.mk in OpenBSD (dated Jun 3 > 22:47:10 1996 UTC) already said > > LOCALBASE ?= /usr/local > PREFIX ?= ${LOCALBASE} > [...] >> I had a discussion in NGINX Unit about it, and >> the decission for now has been: "support prefix=/usr/local for default >> manual installation through the Makefile, and let BSD users adjust to >> their preferred path". > > That's an *excellent* solution for the task, thanks for doing it > the right way. By setting PREFIX=/usr/local by default in the > upstream Makefile, you are minimizing the work for *BSD porters. > > The BSD ports frameworks will typically run the upstreak "make install" > with the variable DESTDIR set to a custom value, for example > > DESTDIR=/usr/ports/pobj/groff-1.23.0/fake-amd64 > > so if the upstream Makefile sets PREFIX=/usr/local , > that's perfect, everything gets installed to the right place > without an intervention by the person doing the porting. > > Of course, if the upstream Makefile would use some other PREFIX, > that would not be a huge obstacle. All we have to do in that case > is pass the option --prefix=/usr/local to the ./configure script, > or something equivalent if the software isn't using GNU configure. > >> We were concerned that we might get collisions >> with the BSD port also installing in /usr/local, but that's the least >> evil (and considering BSD users don't typically run `make install`, it's >> not so bad). > > It's not bad at all. It's perfect. > > Of course, if a user wants to install *without* the ports framework, > they have to provide their own --prefix. But that's not an issue > because it is easy to do, and installing without a port is discouraged > anyway.
===
Directory variables should never contain a trailing slash (I've learned that the hard way, where some things would break unexpectedly). Especially, make(1) is likely to have problems when things have double slashes or a trailing slash, since it treats filenames as text strings. I've removed the trailing slash from the prefix, and added it to the derivate variables just after the prefix. pkg-config(1) also expects directory variables to have no trailing slash.
===
I also removed the code that would set variables as depending on the prefix if they didn't start with a slash, because that is a rather non-obvious behavior, and things should not always depend on prefix, but other dirs such as $(runstatedir), so if we keep a similar behavior it would be very unreliable. Better keep variables intact if set, or use the default if unset.
===
Print the real defaults for ./configure --help, rather than the actual values.
===
I used a subdirectory under the standard /var/lib for NXT_STATE, instead of a homemade "state" dir that does the same thing.
===
Modified the Makefile to create some dirs that weren't being created, and also remove those that weren't being removed in uninstall, probably because someone forgot to add them.
===
Add new options for setting the new variables, and rename some to be consistent with the standard names. Keep the old ones at configuration time for compatibility, but mark them as deprecated. Don't keep the old ones at exec time.
===
A summary of the default config is:
Unit configuration summary:
bin directory: ............. "/usr/local/bin" sbin directory: ............ "/usr/local/sbin" lib directory: ............. "/usr/local/lib" include directory: ......... "/usr/local/include" man pages directory: ....... "/usr/local/share/man" modules directory: ......... "/usr/local/lib/unit/modules" state directory: ........... "/usr/local/var/lib/unit" tmp directory: ............. "/tmp"
pid file: .................. "/usr/local/var/run/unit/unit.pid" log file: .................. "/usr/local/var/log/unit/unit.log"
control API socket: ........ "unix:/usr/local/var/run/unit/control.unit.sock"
Link: <https://www.gnu.org/prep/standards/html_node/Directory-Variables.html> Link: <https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html> Reviewed-by: Artem Konev <a.konev@f5.com> Reviewed-by: Andrew Clayton <a.clayton@nginx.com> Tested-by: Andrew Clayton <a.clayton@nginx.com> Reviewed-by: Konstantin Pavlov <thresh@nginx.com> Signed-off-by: Alejandro Colomar <alx@nginx.com> diff 2314:bc5a90e2e6e8 Thu Jul 14 11:25:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Added default values for pathnames.
This allows one to simply run `./configure` and expect it to produce sane defaults for an install.
Previously, without specifying `--prefix=...`, `make install` would simply fail, recommending to set `--prefix` or `DESTDIR`, but that recommendation was incomplete at best, since it didn't set many of the subdirs needed for a good organization.
Setting `DESTDIR` was even worse, since that shouldn't even affect an installation (it is required to be transparent to the installation).
/usr/local is the historic Unix standard path to use for installations from source made manually by the admin of the system. Some package managers (Homebrew, I'm looking specifically at you) have abused that path to install their things, but 1) it's not our fault that someone else incorrectly abuses that path (and they seem to be fixing it for newer archs; e.g., they started using /opt/homebrew for Apple Silicon), 2) there's no better path than /usr/local, 3) we still allow changing it for systems where this might not be the desired path (MacOS Intel with hombrew), and 4) it's _the standard_.
See a related conversation with Ingo (OpenBSD maintainer):
On 7/27/22 16:16, Ingo Schwarze wrote: > Hi Alejandro, [...] > > Alejandro Colomar wrote on Sun, Jul 24, 2022 at 07:07:18PM +0200: >> On 7/24/22 16:57, Ingo Schwarze wrote: >>> Alejandro Colomar wrote on Sun, Jul 24, 2022 at 01:20:46PM +0200: > >>>> /usr/local is for sysadmins to build from source; > >>> Doing that is *very* strongly discouraged on OpenBSD. > >> I guess that's why the directory was reused in the BSDs to install ports >> (probably ports were installed by the sysadmin there, and by extension, >> ports are now always installed there, but that's just a guess). > > Maybe. In any case, the practice of using /usr/local for packages > created from ports is significantly older than the recommendation > to refrain from using upstream "make install" outside the ports > framework. > > * The FreeBSD ports framework was started by Jordan Hubbard in 1993. > * The ports framework was ported from FreeBSD to OpenBSD > by Niklas Hallqvist in 1996. > * NetBSD pkgsrc was forked from FreeBSD ports by Alistair G. Crooks > and Hubert Feyrer in 1997. > > I failed to quickly find Jordan's original version, but rev. 1.1 > of /usr/ports/infrastructure/mk/bsd.port.mk in OpenBSD (dated Jun 3 > 22:47:10 1996 UTC) already said > > LOCALBASE ?= /usr/local > PREFIX ?= ${LOCALBASE} > [...] >> I had a discussion in NGINX Unit about it, and >> the decission for now has been: "support prefix=/usr/local for default >> manual installation through the Makefile, and let BSD users adjust to >> their preferred path". > > That's an *excellent* solution for the task, thanks for doing it > the right way. By setting PREFIX=/usr/local by default in the > upstream Makefile, you are minimizing the work for *BSD porters. > > The BSD ports frameworks will typically run the upstreak "make install" > with the variable DESTDIR set to a custom value, for example > > DESTDIR=/usr/ports/pobj/groff-1.23.0/fake-amd64 > > so if the upstream Makefile sets PREFIX=/usr/local , > that's perfect, everything gets installed to the right place > without an intervention by the person doing the porting. > > Of course, if the upstream Makefile would use some other PREFIX, > that would not be a huge obstacle. All we have to do in that case > is pass the option --prefix=/usr/local to the ./configure script, > or something equivalent if the software isn't using GNU configure. > >> We were concerned that we might get collisions >> with the BSD port also installing in /usr/local, but that's the least >> evil (and considering BSD users don't typically run `make install`, it's >> not so bad). > > It's not bad at all. It's perfect. > > Of course, if a user wants to install *without* the ports framework, > they have to provide their own --prefix. But that's not an issue > because it is easy to do, and installing without a port is discouraged > anyway.
===
Directory variables should never contain a trailing slash (I've learned that the hard way, where some things would break unexpectedly). Especially, make(1) is likely to have problems when things have double slashes or a trailing slash, since it treats filenames as text strings. I've removed the trailing slash from the prefix, and added it to the derivate variables just after the prefix. pkg-config(1) also expects directory variables to have no trailing slash.
===
I also removed the code that would set variables as depending on the prefix if they didn't start with a slash, because that is a rather non-obvious behavior, and things should not always depend on prefix, but other dirs such as $(runstatedir), so if we keep a similar behavior it would be very unreliable. Better keep variables intact if set, or use the default if unset.
===
Print the real defaults for ./configure --help, rather than the actual values.
===
I used a subdirectory under the standard /var/lib for NXT_STATE, instead of a homemade "state" dir that does the same thing.
===
Modified the Makefile to create some dirs that weren't being created, and also remove those that weren't being removed in uninstall, probably because someone forgot to add them.
===
Add new options for setting the new variables, and rename some to be consistent with the standard names. Keep the old ones at configuration time for compatibility, but mark them as deprecated. Don't keep the old ones at exec time.
===
A summary of the default config is:
Unit configuration summary:
bin directory: ............. "/usr/local/bin" sbin directory: ............ "/usr/local/sbin" lib directory: ............. "/usr/local/lib" include directory: ......... "/usr/local/include" man pages directory: ....... "/usr/local/share/man" modules directory: ......... "/usr/local/lib/unit/modules" state directory: ........... "/usr/local/var/lib/unit" tmp directory: ............. "/tmp"
pid file: .................. "/usr/local/var/run/unit/unit.pid" log file: .................. "/usr/local/var/log/unit/unit.log"
control API socket: ........ "unix:/usr/local/var/run/unit/control.unit.sock"
Link: <https://www.gnu.org/prep/standards/html_node/Directory-Variables.html> Link: <https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html> Reviewed-by: Artem Konev <a.konev@f5.com> Reviewed-by: Andrew Clayton <a.clayton@nginx.com> Tested-by: Andrew Clayton <a.clayton@nginx.com> Reviewed-by: Konstantin Pavlov <thresh@nginx.com> Signed-off-by: Alejandro Colomar <alx@nginx.com> diff 2314:bc5a90e2e6e8 Thu Jul 14 11:25:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Added default values for pathnames.
This allows one to simply run `./configure` and expect it to produce sane defaults for an install.
Previously, without specifying `--prefix=...`, `make install` would simply fail, recommending to set `--prefix` or `DESTDIR`, but that recommendation was incomplete at best, since it didn't set many of the subdirs needed for a good organization.
Setting `DESTDIR` was even worse, since that shouldn't even affect an installation (it is required to be transparent to the installation).
/usr/local is the historic Unix standard path to use for installations from source made manually by the admin of the system. Some package managers (Homebrew, I'm looking specifically at you) have abused that path to install their things, but 1) it's not our fault that someone else incorrectly abuses that path (and they seem to be fixing it for newer archs; e.g., they started using /opt/homebrew for Apple Silicon), 2) there's no better path than /usr/local, 3) we still allow changing it for systems where this might not be the desired path (MacOS Intel with hombrew), and 4) it's _the standard_.
See a related conversation with Ingo (OpenBSD maintainer):
On 7/27/22 16:16, Ingo Schwarze wrote: > Hi Alejandro, [...] > > Alejandro Colomar wrote on Sun, Jul 24, 2022 at 07:07:18PM +0200: >> On 7/24/22 16:57, Ingo Schwarze wrote: >>> Alejandro Colomar wrote on Sun, Jul 24, 2022 at 01:20:46PM +0200: > >>>> /usr/local is for sysadmins to build from source; > >>> Doing that is *very* strongly discouraged on OpenBSD. > >> I guess that's why the directory was reused in the BSDs to install ports >> (probably ports were installed by the sysadmin there, and by extension, >> ports are now always installed there, but that's just a guess). > > Maybe. In any case, the practice of using /usr/local for packages > created from ports is significantly older than the recommendation > to refrain from using upstream "make install" outside the ports > framework. > > * The FreeBSD ports framework was started by Jordan Hubbard in 1993. > * The ports framework was ported from FreeBSD to OpenBSD > by Niklas Hallqvist in 1996. > * NetBSD pkgsrc was forked from FreeBSD ports by Alistair G. Crooks > and Hubert Feyrer in 1997. > > I failed to quickly find Jordan's original version, but rev. 1.1 > of /usr/ports/infrastructure/mk/bsd.port.mk in OpenBSD (dated Jun 3 > 22:47:10 1996 UTC) already said > > LOCALBASE ?= /usr/local > PREFIX ?= ${LOCALBASE} > [...] >> I had a discussion in NGINX Unit about it, and >> the decission for now has been: "support prefix=/usr/local for default >> manual installation through the Makefile, and let BSD users adjust to >> their preferred path". > > That's an *excellent* solution for the task, thanks for doing it > the right way. By setting PREFIX=/usr/local by default in the > upstream Makefile, you are minimizing the work for *BSD porters. > > The BSD ports frameworks will typically run the upstreak "make install" > with the variable DESTDIR set to a custom value, for example > > DESTDIR=/usr/ports/pobj/groff-1.23.0/fake-amd64 > > so if the upstream Makefile sets PREFIX=/usr/local , > that's perfect, everything gets installed to the right place > without an intervention by the person doing the porting. > > Of course, if the upstream Makefile would use some other PREFIX, > that would not be a huge obstacle. All we have to do in that case > is pass the option --prefix=/usr/local to the ./configure script, > or something equivalent if the software isn't using GNU configure. > >> We were concerned that we might get collisions >> with the BSD port also installing in /usr/local, but that's the least >> evil (and considering BSD users don't typically run `make install`, it's >> not so bad). > > It's not bad at all. It's perfect. > > Of course, if a user wants to install *without* the ports framework, > they have to provide their own --prefix. But that's not an issue > because it is easy to do, and installing without a port is discouraged > anyway.
===
Directory variables should never contain a trailing slash (I've learned that the hard way, where some things would break unexpectedly). Especially, make(1) is likely to have problems when things have double slashes or a trailing slash, since it treats filenames as text strings. I've removed the trailing slash from the prefix, and added it to the derivate variables just after the prefix. pkg-config(1) also expects directory variables to have no trailing slash.
===
I also removed the code that would set variables as depending on the prefix if they didn't start with a slash, because that is a rather non-obvious behavior, and things should not always depend on prefix, but other dirs such as $(runstatedir), so if we keep a similar behavior it would be very unreliable. Better keep variables intact if set, or use the default if unset.
===
Print the real defaults for ./configure --help, rather than the actual values.
===
I used a subdirectory under the standard /var/lib for NXT_STATE, instead of a homemade "state" dir that does the same thing.
===
Modified the Makefile to create some dirs that weren't being created, and also remove those that weren't being removed in uninstall, probably because someone forgot to add them.
===
Add new options for setting the new variables, and rename some to be consistent with the standard names. Keep the old ones at configuration time for compatibility, but mark them as deprecated. Don't keep the old ones at exec time.
===
A summary of the default config is:
Unit configuration summary:
bin directory: ............. "/usr/local/bin" sbin directory: ............ "/usr/local/sbin" lib directory: ............. "/usr/local/lib" include directory: ......... "/usr/local/include" man pages directory: ....... "/usr/local/share/man" modules directory: ......... "/usr/local/lib/unit/modules" state directory: ........... "/usr/local/var/lib/unit" tmp directory: ............. "/tmp"
pid file: .................. "/usr/local/var/run/unit/unit.pid" log file: .................. "/usr/local/var/log/unit/unit.log"
control API socket: ........ "unix:/usr/local/var/run/unit/control.unit.sock"
Link: <https://www.gnu.org/prep/standards/html_node/Directory-Variables.html> Link: <https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html> Reviewed-by: Artem Konev <a.konev@f5.com> Reviewed-by: Andrew Clayton <a.clayton@nginx.com> Tested-by: Andrew Clayton <a.clayton@nginx.com> Reviewed-by: Konstantin Pavlov <thresh@nginx.com> Signed-off-by: Alejandro Colomar <alx@nginx.com> diff 2262:0bca3af03c75 Mon Oct 24 16:14:00 UTC 2022 Andrew Clayton <a.clayton@nginx.com> Isolation: wired up cgroup support to the config system.
This hooks the cgroup support up to the config system so it can actually be used.
To make use of this in unit a new "cgroup" section has been added to the isolation configuration.
e.g
"applications": { "python": { "type": "python", "processes": 5, "path": "/opt/unit/unit-cgroup-test/", "module": "app",
"isolation": { "cgroup": { "path": "app/python" } } } }
Now there are two ways to specify the path, relative, like the above (without a leading '/') and absolute (with a leading '/').
In the above case the "python" application is placed into its own cgroup under CGROUP_ROOT/<main unit process cgroup>/app/python. Whereas if you specified say
"path": "/unit/app/python"
Then the python application would be placed under CGROUP_ROOT/unit/app/python
The first option allows you to easily take advantage of any resource limits that have already been configured for unit.
With the second method (absolute pathname) if you know of an already existing cgroup where you'd like to place it, you can, e.g
"path": "/system.slice/unit/python"
Where system.slice has already been created by systemd and may already have some overall system limits applied which would also apply to unit. Limits apply down the hierarchy and lower groups can't exceed the previous group limits.
So what does this actually look like? Lets take the unit-calculator application[0] and have each of its applications placed into their own cgroup. If we give each application a new section like
"isolation": { "cgroup": { "path": "/unit/unit-calculator/add" } }
changing the path for each one, we can visualise the result with the systemd-cgls command, e.g
│ └─session-5.scope (#4561) │ ├─ 6667 sshd: andrew [priv] │ ├─ 6684 sshd: andrew@pts/0 │ ├─ 6685 -bash │ ├─ 12632 unit: main v1.28.0 [/opt/unit/sbin/unitd --control 127.0.0.1:808> │ ├─ 12634 unit: controller │ ├─ 12635 unit: router │ ├─ 13550 systemd-cgls │ └─ 13551 less ├─unit (#4759) │ └─unit-calculator (#5037) │ ├─subtract (#5069) │ │ ├─ 12650 unit: "subtract" prototype │ │ └─ 12651 unit: "subtract" application │ ├─multiply (#5085) │ │ ├─ 12653 unit: "multiply" prototype │ │ └─ 12654 unit: "multiply" application │ ├─divide (#5101) │ │ ├─ 12671 unit: "divide" prototype │ │ └─ 12672 node divide.js │ ├─sqroot (#5117) │ │ ├─ 12679 unit: "sqroot" prototype │ │ └─ 12680 /home/andrew/src/unit-calculator/sqroot/sqroot │ └─add (#5053) │ ├─ 12648 unit: "add" prototype │ └─ 12649 unit: "add" application
We used an absolute path so the cgroups will be created relative to the main cgroupfs mount, e.g /sys/fs/cgroup
We can see that the main unit processes are in the same cgroup as the shell from where they were started, by default child process are placed into the same cgroup as the parent.
Then we can see that each application has been placed into its own cgroup under /sys/fs/cgroup
Taking another example of a simple 5 process python application, with
"isolation": { "cgroup": { "path": "app/python" } }
Here we have specified a relative path and thus the python application will be placed below the existing cgroup that contains the main unit process. E.g
│ │ │ ├─app-glib-cinnamon\x2dcustom\x2dlauncher\x2d3-43951.scope (#90951) │ │ │ │ ├─ 988 unit: main v1.28.0 [/opt/unit/sbin/unitd --no-daemon] │ │ │ │ ├─ 990 unit: controller │ │ │ │ ├─ 991 unit: router │ │ │ │ ├─ 43951 xterm -bg rgb:20/20/20 -fg white -fa DejaVu Sans Mono │ │ │ │ ├─ 43956 bash │ │ │ │ ├─ 58828 sudo -i │ │ │ │ ├─ 58831 -bash │ │ │ │ └─app (#107351) │ │ │ │ └─python (#107367) │ │ │ │ ├─ 992 unit: "python" prototype │ │ │ │ ├─ 993 unit: "python" application │ │ │ │ ├─ 994 unit: "python" application │ │ │ │ ├─ 995 unit: "python" application │ │ │ │ ├─ 996 unit: "python" application │ │ │ │ └─ 997 unit: "python" application
[0]: <https://github.com/lcrilly/unit-calculator>
Reviewed-by: Alejandro Colomar <alx@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com> diff 2145:ee1e319c8ed8 Sun Apr 24 22:57:00 UTC 2022 Timo Stark <tippexs91@googlemail.com> Increased readtimeout for configuration endpoint.
Closes: <https://github.com/nginx/unit/issues/676> diff 1951:863363011a94 Tue Aug 24 11:38:00 UTC 2021 Valentin Bartenev <vbart@nginx.com> Version bump. diff 1876:f85b85094541 Mon May 24 09:01:00 UTC 2021 Oisin Canty <o.canty@f5.com> Node.js: renamed "require_shim" to "loader".
|
/unit/src/ |
H A D | nxt_unit.c | diff 2014:f8a0992944df Wed Nov 24 10:11:00 UTC 2021 Max Romanov <max.romanov@nginx.com> Sending shared port to application prototype.
Application process started with shared port (and queue) already configured. But still waits for PORT_ACK message from router to start request processing (so-called "ready state").
Waiting for router confirmation is necessary. Otherwise, the application may produce response and send it to router before the router have the information about the application process. This is a subject of further optimizations. diff 1728:b39918d13444 Tue Nov 24 13:40:00 UTC 2020 Valentin Bartenev <vbart@nginx.com> Libunit: improved error logging around initialization env variable. diff 1321:2c7f79bf0a1f Tue Dec 24 15:04:00 UTC 2019 Max Romanov <max.romanov@nginx.com> Introducing port messages to notify about out of shared memory.
- OOSM (out of shared memory). Sent by application process to router when application reaches the limit of allocated shared memory and needs more. - SHM_ACK. Sent by router to application when the application's shared memory is released and the OOSM flag is enabled for the segment.
This implements blocking mode (the library waits for SHM_ACK in case of out of shared memory condition and retries allocating the required memory amount) and non-blocking mode (the library notifies the application that it's out of shared memory and returns control to the application module that sets up the output queue and puts SHM_ACK in the main message loop). diff 1320:4e70411b9842 Tue Dec 24 15:04:00 UTC 2019 Max Romanov <max.romanov@nginx.com> Adding "limits/shm" configuration validation and parsing. diff 1319:706e07600a26 Tue Dec 24 15:04:00 UTC 2019 Max Romanov <max.romanov@nginx.com> Renaming nxt_unit_mmap_buf_remove to nxt_unit_mmap_buf_unlink.
The function unchains the buffer from the buffer's linked list. diff 1317:a098368e5364 Tue Dec 24 15:03:00 UTC 2019 Max Romanov <max.romanov@nginx.com> Using non-shared memory buffers for small messages.
Current shared memory buffer implementation uses fixed-size memory blocks, allocating at least 16384 bytes. When application sends data in a large number of small chunks, it makes sense to buffer them or use plain memory buffers to improve performance and reduce memory footprint.
This patch introduces minimum size limit (1024 bytes) for shared memory buffers.
|
H A D | nxt_application.c | diff 2014:f8a0992944df Wed Nov 24 10:11:00 UTC 2021 Max Romanov <max.romanov@nginx.com> Sending shared port to application prototype.
Application process started with shared port (and queue) already configured. But still waits for PORT_ACK message from router to start request processing (so-called "ready state").
Waiting for router confirmation is necessary. Otherwise, the application may produce response and send it to router before the router have the information about the application process. This is a subject of further optimizations. diff 1518:46e8c414db88 Thu Jul 23 11:24:00 UTC 2020 Max Romanov <max.romanov@nginx.com> Fixing main and application port structs file descriptor init.
Correct value for non-initialized file descriptor is -1, because most of the checks in libunit compares file descriptor with -1 before performing an action. Using 0 as default value, may cause to close file descriptor #0, this may affect application logic.
It is not required to list this patch in changelog because impact is not seen by end users. diff 494:7c83ddcc1c42 Wed Jan 24 12:16:00 UTC 2018 Sergey Kandaurov <pluknet@nginx.com> Fixed formatting in nxt_sprintf() and logging. diff 493:745222d540a2 Wed Jan 24 12:16:00 UTC 2018 Sergey Kandaurov <pluknet@nginx.com> Using size_t for the field width type of the "%*s" specifier.
|
H A D | nxt_runtime.h | diff 223:bf98efe2c55c Thu Aug 24 14:43:00 UTC 2017 Igor Sysoev <igor@sysoev.ru> Removed configure option --no-threads. diff 220:3b86550821c5 Thu Aug 24 14:29:00 UTC 2017 Igor Sysoev <igor@sysoev.ru> Removed command line option --workers.
|
H A D | nxt_clone.c | diff 2629:116cb969f351 Wed Jan 24 18:01:00 UTC 2024 Andrew Clayton <a.clayton@nginx.com> Isolation: Use an appropriate type for storing uid/gids
Andrei reported an issue on arm64 where he was seeing the following error message when running the tests
2024/01/17 18:32:31.109 [error] 54904#54904 "gidmap" field has an entry with "size": 1, but for unprivileged unit it must be 1.
This error message is guarded by the following if statement
if (nxt_slow_path(m.size > 1)
Turns out size was indeed > 1, in this case it was 289356276058554369, m.size is defined as a nxt_int_t, which on arm64 is actually 8 bytes, but was being printed as a signed int (4 bytes) and by chance/undefined behaviour comes out as 1.
But why is size so big? In this case it should have just been 1 with a config of
'gidmap': [{'container': 0, 'host': os.getegid(), 'size': 1}],
This is due to nxt_int_t being 64bits on arm64 but using a conf type of NXT_CONF_MAP_INT which means in nxt_conf_map_object() we would do (using our m.size variable as an example)
ptr = nxt_pointer_to(data, map[i].offset); ... ptr->i = num;
Where ptr is a union pointer and is now pointing at our m.size
Next we set m.size to the value of num (which is 1 in this case), via ptr->i where i is a member of that union of type int.
So here we are setting a 64bit memory location (nxt_int_t on arm64) through a 32bit (int) union alias, this means we are only setting the lower half (4) of the bytes.
Whatever happens to be in the upper 4 bytes will remain, giving us our exceptionally large value.
This is demonstrated by this program
#include <stdio.h> #include <stdint.h>
int main(void) { int64_t num = -1; /* All 1's in two's complement */ union { int32_t i32; int64_t i64; } *ptr;
ptr = (void *)#
ptr->i32 = 1; printf("num : %lu / %ld\n", num, num); ptr->i64 = 1; printf("num : %ld\n", num);
return 0; } $ make union-32-64-issue cc union-32-64-issue.c -o union-32-64-issue $ ./union-32-64-issue num : 18446744069414584321 / -4294967295 num : 1
However that is not the only issue, because the members of nxt_clone_map_entry_t were specified as nxt_int_t's on the likes of x86_64 this would be a 32bit signed integer. However uid/gids on Linux at least are defined as unsigned integers, so a nxt_int_t would not be big enough to hold all potential values.
We could make the nxt_uint_t's but then we're back to the above union aliasing problem.
We could just set the memory for these variables to 0 and that would work, however that's really just papering over the problem.
The right thing is to use a large enough sized type to store these things, hence the previously introduced nxt_cred_t. This is an int64_t which is plenty large enough.
So we switch the nxt_clone_map_entry_t structure members over to nxt_cred_t's and use NXT_CONF_MAP_INT64 as the conf type, which then uses the right sized union member in nxt_conf_map_object() to set these variables.
Reported-by: Andrei Zeliankou <zelenkov@nginx.com> Reviewed-by: Zhidao Hong <z.hong@f5.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
H A D | nxt_application.h | diff 2014:f8a0992944df Wed Nov 24 10:11:00 UTC 2021 Max Romanov <max.romanov@nginx.com> Sending shared port to application prototype.
Application process started with shared port (and queue) already configured. But still waits for PORT_ACK message from router to start request processing (so-called "ready state").
Waiting for router confirmation is necessary. Otherwise, the application may produce response and send it to router before the router have the information about the application process. This is a subject of further optimizations. diff 1320:4e70411b9842 Tue Dec 24 15:04:00 UTC 2019 Max Romanov <max.romanov@nginx.com> Adding "limits/shm" configuration validation and parsing. diff 272:24b036fbd431 Tue Sep 05 17:22:00 UTC 2017 Max Romanov <max.romanov@nginx.com> Using CSTRZ mapping type for go executable.
|
H A D | nxt_port_memory.c | diff 1321:2c7f79bf0a1f Tue Dec 24 15:04:00 UTC 2019 Max Romanov <max.romanov@nginx.com> Introducing port messages to notify about out of shared memory.
- OOSM (out of shared memory). Sent by application process to router when application reaches the limit of allocated shared memory and needs more. - SHM_ACK. Sent by router to application when the application's shared memory is released and the OOSM flag is enabled for the segment.
This implements blocking mode (the library waits for SHM_ACK in case of out of shared memory condition and retries allocating the required memory amount) and non-blocking mode (the library notifies the application that it's out of shared memory and returns control to the application module that sets up the output queue and puts SHM_ACK in the main message loop). diff 494:7c83ddcc1c42 Wed Jan 24 12:16:00 UTC 2018 Sergey Kandaurov <pluknet@nginx.com> Fixed formatting in nxt_sprintf() and logging.
|
H A D | nxt_process.h | diff 2260:3005b3de99a5 Mon Oct 24 16:35:00 UTC 2022 Andrew Clayton <a.clayton@nginx.com> Isolation: wired up per-application cgroup support internally.
This commit hooks into the cgroup infrastructure added in the previous commit to create per-application cgroups.
It does this by adding each "prototype process" into its own cgroup, then each child process inherits its parents cgroup.
If we fail to create a cgroup we simply fail the process. This behaviour may get enhanced in the future.
This won't actually do anything yet. Subsequent commits will hook this up to the build and config systems.
Reviewed-by: Alejandro Colomar <alx@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com> diff 2031:e8518399bc10 Wed Nov 24 10:11:00 UTC 2021 Max Romanov <max.romanov@nginx.com> Fixing alerts on router restart.
Splitting the process type connectivity matrix to 'keep ports' and 'send ports'; the 'keep ports' matrix is used to clean up unnecessary ports after forking a new process, and the 'send ports' matrix determines which process types expect to get created process ports.
Unfortunately, the original single connectivity matrix no longer works because of an application stop delay caused by prototypes. Existing applications should not get the new router port at the moment. diff 2015:4570130dd183 Wed Nov 24 10:11:00 UTC 2021 Max Romanov <max.romanov@nginx.com> Fixing alerts on router restart.
Splitting the process type connectivity matrix to 'keep ports' and 'send ports'; the 'keep ports' matrix is used to clean up unnecessary ports after forking a new process, and the 'send ports' matrix determines which process types expect to get created process ports.
Unfortunately, the original single connectivity matrix no longer works because of an application stop delay caused by prototypes. Existing applications should not get the new router port at the moment.
|
H A D | nxt_h1proto.c | diff 1318:f237e8c553fd Tue Dec 24 15:04:00 UTC 2019 Max Romanov <max.romanov@nginx.com> Introducing write tail reference to avoid buffer chain iteration. diff 1110:4ca6df50b4d4 Wed Jul 24 10:47:00 UTC 2019 Axel Duch <axel.duch@nginx.com> Added routing based on request scheme.
Scheme matches exact string “http” or “https”. diff 494:7c83ddcc1c42 Wed Jan 24 12:16:00 UTC 2018 Sergey Kandaurov <pluknet@nginx.com> Fixed formatting in nxt_sprintf() and logging.
|
H A D | nxt_http.h | diff 1925:b8a2ac618950 Sat Jul 24 03:44:00 UTC 2021 Zhidao HONG <z.hong@f5.com> Router: split nxt_http_app_conf_t from nxt_http_action_t.
No functional changes. diff 1903:f17827e75e25 Mon May 24 08:15:00 UTC 2021 Zhidao HONG <z.hong@f5.com> Router: split nxt_http_return_conf_t from nxt_http_action_t.
No functional changes. diff 1110:4ca6df50b4d4 Wed Jul 24 10:47:00 UTC 2019 Axel Duch <axel.duch@nginx.com> Added routing based on request scheme.
Scheme matches exact string “http” or “https”.
|
H A D | nxt_http_return.c | diff 1903:f17827e75e25 Mon May 24 08:15:00 UTC 2021 Zhidao HONG <z.hong@f5.com> Router: split nxt_http_return_conf_t from nxt_http_action_t.
No functional changes.
|
H A D | nxt_port.c | diff 2031:e8518399bc10 Wed Nov 24 10:11:00 UTC 2021 Max Romanov <max.romanov@nginx.com> Fixing alerts on router restart.
Splitting the process type connectivity matrix to 'keep ports' and 'send ports'; the 'keep ports' matrix is used to clean up unnecessary ports after forking a new process, and the 'send ports' matrix determines which process types expect to get created process ports.
Unfortunately, the original single connectivity matrix no longer works because of an application stop delay caused by prototypes. Existing applications should not get the new router port at the moment. diff 2015:4570130dd183 Wed Nov 24 10:11:00 UTC 2021 Max Romanov <max.romanov@nginx.com> Fixing alerts on router restart.
Splitting the process type connectivity matrix to 'keep ports' and 'send ports'; the 'keep ports' matrix is used to clean up unnecessary ports after forking a new process, and the 'send ports' matrix determines which process types expect to get created process ports.
Unfortunately, the original single connectivity matrix no longer works because of an application stop delay caused by prototypes. Existing applications should not get the new router port at the moment. diff 2014:f8a0992944df Wed Nov 24 10:11:00 UTC 2021 Max Romanov <max.romanov@nginx.com> Sending shared port to application prototype.
Application process started with shared port (and queue) already configured. But still waits for PORT_ACK message from router to start request processing (so-called "ready state").
Waiting for router confirmation is necessary. Otherwise, the application may produce response and send it to router before the router have the information about the application process. This is a subject of further optimizations.
|
H A D | nxt_http_static.c | diff 2619:24bba891e97e Wed Jan 17 17:18:00 UTC 2024 Andrew Clayton <a.clayton@nginx.com> HTTP: Remove short read check in nxt_http_static_buf_completion()
On GH, @tonychuuy reported an issue when using Units 'share' action they would get the following error in the unit log
2024/01/15 17:53:41 [error] 49#52 *103 file "/var/www/html/public/vendor/telescope/app.css" has changed while sending response to a client
This would happen when trying to serve files over a certain size and the requested file would not be sent.
This is due to a somewhat bogus check in nxt_http_static_buf_completion()
I say bogus because it's not clear what the check is trying to accomplish and the error message is not entirely accurate either.
The check in question goes like
n = pread(file->fd, buf, size, offset); return n; ... if (n != size) { if (n >= 0) { /* log file changed error and finish */
/* >> Problem is here << */ }
/* log general error and finish */ }
If the number of bytes read is not what we asked for and is > -1 (i.e not an error) then it says the file has changed, but really it only checks if the file has _shrunk_ (we can't get back _more_ bytes than we asked for) since it was stat'd.
This is what happens
recvfrom(22, "GET /tfile HTTP/1.1\r\nHost: local"..., 2048, 0, NULL, NULL) = 82 openat(AT_FDCWD, "/mnt/9p/tfile", O_RDONLY|O_NONBLOCK) = 23 newfstatat(23, "", {st_mode=S_IFREG|0644, st_size=149922, ...}, AT_EMPTY_PATH) = 0
We get a request from a client, open the requested file and stat(2) it to get the file size.
We would then go into a pread/writev loop reading the file data and sending it to the client until it's all been sent.
However what was happening in this case was this (showing a dummy file of 149922 bytes)
pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 131072, 0) = 61440 write(2, "2024/01/17 15:30:50 [error] 1849"..., 109) = 109
We wanted to read 131072 bytes but only read 61440 bytes, the above check triggered and the file transfer was aborted and the above error message logged.
Normally for a regular file you will only get less bytes than asked for if the read call is interrupted by a signal or you're near the end of file.
There is however at least another situation where this may happen, if the file in question is being served from a network filesystem.
It turns out that was indeed the case here, the files where being served over the 9P filesystem protocol. Unit was running in a docker container in an Ubuntu VM under Windows/WSL2 and the files where being passed through to the VM from Windows over 9P.
Whatever the intention of this check, it is clearly causing issues in real world scenarios.
If it was really desired to check if the had changed since it was opened/stat'd then it would require a different methodology and be a patch for another day. But as it stands this current check does more harm than good, so lets just remove it.
With it removed we now get for the above test file
recvfrom(22, "GET /tfile HTTP/1.1\r\nHost: local"..., 2048, 0, NULL, NULL) = 82 openat(AT_FDCWD, "/mnt/9p/tfile", O_RDONLY|O_NONBLOCK) = 23 newfstatat(23, "", {st_mode=S_IFREG|0644, st_size=149922, ...}, AT_EMPTY_PATH) = 0 mmap(NULL, 135168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f367817b000 pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 131072, 0) = 61440 pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 18850, 61440) = 18850 writev(22, [{iov_base="HTTP/1.1 200 OK\r\nLast-Modified: "..., iov_len=171}, {iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=61440}, {iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=18850}], 3) = 80461 pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 69632, 80290) = 61440 pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 8192, 141730) = 8192 close(23) = 0 writev(22, [{iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=61440}, {iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=8192}], 2) = 69632
So we can see we do two pread(2)s's and a writev(2), then another two pread(2)s and another writev(2) and all the file data has been read and sent to the client.
Reported-by: tonychuuy <https://github.com/tonychuuy> Link: <https://en.wikipedia.org/wiki/9P_(protocol)> Fixes: 08a8d1510 ("Basic support for serving static files.") Closes: https://github.com/nginx/unit/issues/1064 Reviewed-by: Zhidao Hong <z.hong@f5.com> Reviewed-by: Andrei Zeliankou <zelenkov@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com> diff 1200:6d0c7bb314b0 Tue Sep 24 12:33:00 UTC 2019 Valentin Bartenev <vbart@nginx.com> Static: returning 404 for Unix domain sockets.
It's now similar to how attempts to access other non-regular files are handled.
|
/unit/test/unit/applications/lang/ |
H A D | node.py | diff 1876:f85b85094541 Mon May 24 09:01:00 UTC 2021 Oisin Canty <o.canty@f5.com> Node.js: renamed "require_shim" to "loader".
|
H A D | java.py | diff 1128:b41fd9ccbe61 Wed Aug 21 11:24:00 UTC 2019 Andrey Zelenkov <zelenkov@nginx.com> Tests: reverted rerun for Java tests in 5e429a7f133c.
Each testcase should also recompile to be recompiled. So backed out for now.
|
/unit/test/ |
H A D | test_go_application.py | diff 1467:195fe0a92670 Fri Apr 24 04:08:00 UTC 2020 Andrei Zeliankou <zelenkov@nginx.com> Tests: introduced module version specification in prerequisites.
|
H A D | test_configuration.py | diff 1467:195fe0a92670 Fri Apr 24 04:08:00 UTC 2020 Andrei Zeliankou <zelenkov@nginx.com> Tests: introduced module version specification in prerequisites. diff 495:879868522dbf Wed Jan 24 12:43:00 UTC 2018 Andrey Zelenkov <zelenkov@nginx.com> Tests: using "expectedFailure" decorator instead of assertTry().
|
H A D | test_go_isolation.py | diff 1467:195fe0a92670 Fri Apr 24 04:08:00 UTC 2020 Andrei Zeliankou <zelenkov@nginx.com> Tests: introduced module version specification in prerequisites.
|
H A D | test_static.py | diff 1204:f0fb4ba90a8e Tue Sep 24 13:13:00 UTC 2019 Andrey Zelenkov <zelenkov@nginx.com> Tests: static tests with file system objects.
|
H A D | test_proxy.py | diff 1467:195fe0a92670 Fri Apr 24 04:08:00 UTC 2020 Andrei Zeliankou <zelenkov@nginx.com> Tests: introduced module version specification in prerequisites.
|
H A D | test_access_log.py | diff 2627:36afea93a678 Wed Jan 24 16:09:00 UTC 2024 Andrei Zeliankou <zelenkov@nginx.com> Tests: "if" option in access logging.
Conditional access logging was introduced here: https://github.com/nginx/unit/commit/4c91bebb50d06b28e369d68b23022caa072cf62d diff 1467:195fe0a92670 Fri Apr 24 04:08:00 UTC 2020 Andrei Zeliankou <zelenkov@nginx.com> Tests: introduced module version specification in prerequisites.
|
/unit/auto/ |
H A D | sources | diff 2261:497ee3475332 Mon Oct 24 13:13:00 UTC 2022 Andrew Clayton <a.clayton@nginx.com> Isolation: wired up cgroup to build system.
This commit enables the building of the cgroup code. This is only built when the cgroupv2 filesystem is found.
If cgroupv2 support is found then
cgroupv2: .................. YES
will be printed by ./configure
Reviewed-by: Alejandro Colomar <alx@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com> diff 1324:73562b05bf48 Tue Dec 24 13:58:00 UTC 2019 Axel Duch <axel.duch@nginx.com> Router: introducing routing on client address. diff 223:bf98efe2c55c Thu Aug 24 14:43:00 UTC 2017 Igor Sysoev <igor@sysoev.ru> Removed configure option --no-threads.
|
/unit/src/ruby/ |
H A D | nxt_ruby.c | diff 2608:2091b078cdf6 Mon Oct 23 13:24:00 UTC 2023 Andrew Clayton <a.clayton@nginx.com> Ruby: Handle response field arrays
@xeron on GitHub reported an issue whereby with a Rails 7.1 application they were getting the following error
2023/10/22 20:57:28 [error] 56#56 [unit] #8: Ruby: Wrong header entry 'value' from application 2023/10/22 20:57:28 [error] 56#56 [unit] #8: Ruby: Failed to run ruby script
After some back and forth debugging it turns out rack was trying to send back a header comprised of an array of values. E.g
app = Proc.new do |env| ["200", { "Content-Type" => "text/plain", "X-Array-Header" => ["Item-1", "Item-2"], }, ["Hello World\n"]] end
run app
It seems this became a possibility in rack v3.0[0]
So along with a header value type of T_STRING we need to also allow T_ARRAY.
If we get a T_ARRAY we need to build up the header field using the given values.
E.g
"X-Array-Header" => ["Item-1", "", "Item-3", "Item-4"],
becomes
X-Array-Header: Item-1; ; Item-3; Item-4
[0]: <https://github.com/rack/rack/blob/main/UPGRADE-GUIDE.md?plain=1#L26>
Reported-by: Ivan Larionov <xeron.oskom@gmail.com> Closes: <https://github.com/nginx/unit/issues/974> Link: <https://github.com/nginx/unit/pull/998> Tested-by: Timo Stark <t.stark@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com> diff 1320:4e70411b9842 Tue Dec 24 15:04:00 UTC 2019 Max Romanov <max.romanov@nginx.com> Adding "limits/shm" configuration validation and parsing.
|