HTTP: controlling response headers support.

NJS: supported loadable modules.

HTTP: added basic URI rewrite.

This commit introduced the basic URI rewrite. It allows users to change request URI. Note the "rewrite" option ignores the contained query if any and the query from th

HTTP: added basic URI rewrite.

This commit introduced the basic URI rewrite. It allows users to change request URI. Note the "rewrite" option ignores the contained query if any and the query from the request is preserverd.
An example:
"routes": [
{
"match": {
"uri": "/v1/test"
},
"action": {
"return": 200
}
},
{
"action": {
"rewrite": "/v1$uri",
"pass": "routes"
}
}
]

Reviewed-by: Alejandro Colomar <alx@nginx.com>

show more ...

Remove a bunch of dead code.

This removes a bunch of unused files that would have been touched by
subsequent commits that switch to using nxt_bool_t (AKA unit6_t) in
structures.

In auto/sources we

Remove a bunch of dead code.

This removes a bunch of unused files that would have been touched by
subsequent commits that switch to using nxt_bool_t (AKA unit6_t) in
structures.

In auto/sources we have

NXT_LIB_SRC0=" \
src/nxt_buf_filter.c \
src/nxt_job_file.c \
src/nxt_stream_module.c \
src/nxt_stream_source.c \
src/nxt_upstream_source.c \
src/nxt_http_source.c \
src/nxt_fastcgi_source.c \
src/nxt_fastcgi_record_parse.c \
\
src/nxt_mem_pool_cleanup.h \
src/nxt_mem_pool_cleanup.c \
"

None of these seem to actually be used anywhere (other than within
themselves). That variable is _not_ referenced anywhere else.

Also remove the unused related header files: src/nxt_buf_filter.h,
src/nxt_fastcgi_source.h, src/nxt_http_source.h, src/nxt_job_file.h,
src/nxt_stream_source.h and src/nxt_upstream_source.h

Also, these files do not seem to be used, no mention under auto/ or build/

src/nxt_file_cache.c
src/nxt_cache.c
src/nxt_job_file_cache.c

src/nxt_cache.h is #included in src/nxt_main.h, but AFAICT is not
actually used.

With all the above removed

$ ./configure --openssl --debug --tests && make -j && make -j tests &&
make libnxt

all builds.

Buildbot passes.

NOTE: You may need to do a 'make clean' before the next build attempt.

Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>

show more ...

Isolation: Rename NXT_HAVE_CLONE -> NXT_HAVE_LINUX_NS.

Due to the need to replace our use of clone/__NR_clone on Linux with
fork(2)/unshare(2) for enabling Linux namespaces(7) to keep the
pthreads(7

Isolation: Rename NXT_HAVE_CLONE -> NXT_HAVE_LINUX_NS.

Due to the need to replace our use of clone/__NR_clone on Linux with
fork(2)/unshare(2) for enabling Linux namespaces(7) to keep the
pthreads(7) API working. Let's rename NXT_HAVE_CLONE to
NXT_HAVE_LINUX_NS, i.e name it after the feature, not how it's
implemented, then in future if we change how we do namespaces again we
don't have to rename this.

Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>

show more ...

Isolation: wired up cgroup to build system.

This commit enables the building of the cgroup code. This is only built
when the cgroupv2 filesystem is found.

If cgroupv2 support is found then

cgrou

Isolation: wired up cgroup to build system.

This commit enables the building of the cgroup code. This is only built
when the cgroupv2 filesystem is found.

If cgroupv2 support is found then

cgroupv2: .................. YES

will be printed by ./configure

Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>

show more ...

NJS: added http request prototype.

Basic njs support.

Var: separating nxt_tstr_t from nxt_var_t.

It's for the introduction of njs support.
For each option that supports native variable and JS template literals introduced next,
it's unified as template

Var: separating nxt_tstr_t from nxt_var_t.

It's for the introduction of njs support.
For each option that supports native variable and JS template literals introduced next,
it's unified as template string.

No functional changes.

show more ...

Fixed the build on MacOS (and others).

@alejandro-colomar reported that the build was broken on MacOS

cc -o build/unitd -pipe -fPIC -fvisibility=hidden -O -W -Wall -Wextra -Wno-unused-parameter -W

Fixed the build on MacOS (and others).

@alejandro-colomar reported that the build was broken on MacOS

cc -o build/unitd -pipe -fPIC -fvisibility=hidden -O -W -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -fstrict-aliasing -Wstrict-overflow=5 -Wmissing-prototypes -Werror -g \
build/src/nxt_main.o build/libnxt.a \
\
\
-L/usr/local/Cellar/pcre2/10.40/lib -lpcre2-8
Undefined symbols for architecture x86_64:
"_nxt_fs_mkdir_parent", referenced from:
_nxt_runtime_pid_file_create in libnxt.a(nxt_runtime.o)
_nxt_runtime_controller_socket in libnxt.a(nxt_controller.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [build/unitd] Error 1

This was due to commit 57fc920 ("Socket: Created control socket & pid file
directories.").

This happened because this commit introduced the usage of
nxt_fs_mkdir_parent() in core code which uses nxt_fs_mkdir(), both of
these are defined in src/nxt_fs.c. It turns out however that this file
doesn't get built on MacOS (or any system that isn't Linux or that
lacks a FreeBSD compatible nmount(2) system call) due to the following

In auto/sources we have

if [ $NXT_HAVE_ROOTFS = YES ]; then
NXT_LIB_SRCS="$NXT_LIB_SRCS src/nxt_fs.c"
fi

NXT_HAVE_ROOTFS is set in auto/isolation

If [ $NXT_HAVE_MOUNT = YES -a $NXT_HAVE_UNMOUNT = YES ]; then
NXT_HAVE_ROOTFS=YES

cat << END >> $NXT_AUTO_CONFIG_H
#ifndef NXT_HAVE_ISOLATION_ROOTFS
#define NXT_HAVE_ISOLATION_ROOTFS 1
#endif
END

fi

While we do have a check for a generic umount(2) which is found on
MacOS, for mount(2) we currently only check for the Linux mount(2) and
FreeBSD nmount(2) system calls. So NXT_HAVE_ROOTFS is set to NO on MacOS
and we don't build src/nxt_fs.c

This fixes the immediate build issue by taking the mount/umount OS
support out of nxt_fs.c into a new nxt_fs_mount.c file which is guarded
by the above while we now build nxt_fs.c unconditionally.

This should fix the build on any _supported_ system.

Reported-by: Alejandro Colomar <alx@nginx.com>
Fixes: 57fc920 ("Socket: Created control socket & pid file directories.")
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>

show more ...

Implemented basic statistics API.

Removed dead code.

nxt_sockaddr_ntop() stopped being used in commit (git) 029942f4eb71.
It has been replaced mostly by nxt_sockaddr_text().

commit 029942f4eb7196c2cff0d0e26bc6ff274138f7d8
A

Removed dead code.

nxt_sockaddr_ntop() stopped being used in commit (git) 029942f4eb71.
It has been replaced mostly by nxt_sockaddr_text().

commit 029942f4eb7196c2cff0d0e26bc6ff274138f7d8
Author: Igor Sysoev <igor@sysoev.ru>
Date: Wed Feb 22 15:09:59 2017 +0300

I/O operations refactoring.

nxt_job_sockaddr_parse() stopped being used in commit (git) 794248090a74.

commit 794248090a74f31cbfcf24ea8c835df2d4d21073
Author: Igor Sysoev <igor@sysoev.ru>
Date: Wed Mar 4 14:04:08 2020 +0300

Legacy upstream code removed.

Also, remove functions and types used only by those two functions:

nxt_job_sockaddr_unix_parse()
nxt_job_sockaddr_inet6_parse()
nxt_job_sockaddr_inet_parse()
nxt_job_sockaddr_parse_t
nxt_job_resolve()
nxt_job_resolve_t

show more ...

Log: split access log from nxt_router.c.

No functional changes.

Introduced SCM_CREDENTIALS / SCM_CREDS in the socket control msgs.

Custom implementation of Base64 decoding function.

Compared to the previous implementation based on OpenSSL, the new implementation
has these advantages:

1. Strict and reliable detection of invali

Custom implementation of Base64 decoding function.

Compared to the previous implementation based on OpenSSL, the new implementation
has these advantages:

1. Strict and reliable detection of invalid strings, including strings with
less than 4 bytes of garbage at the end;

2. Allows to use Base64 strings without '=' padding.

show more ...

Router: matching regular expressions support.

Moved isolation related code to "nxt_isolation.c".

Basic variables support.

Upstream chunked transfer encoding support.

Added "rootfs" feature.

Refactor of process management.

The process abstraction has changed to:

setup(task, process)
start(task, process_data)
prefork(task, process, mp)

The prefork() occurs in the main process rig

Refactor of process management.

The process abstraction has changed to:

setup(task, process)
start(task, process_data)
prefork(task, process, mp)

The prefork() occurs in the main process right before fork.

The file src/nxt_main_process.c is completely free of process
specific logic.

The creation of a process now supports a PROCESS_CREATED state. The
The setup() function of each process can set its state to either
created or ready. If created, a MSG_PROCESS_CREATED is sent to main
process, where external setup can be done (required for rootfs under
container).

The core processes (discovery, controller and router) doesn't need
external setup, then they all proceeds to their start() function
straight away.

In the case of applications, the load of the module happens at the
process setup() time and The module's init() function has changed
to be the start() of the process.

The module API has changed to:

setup(task, process, conf)
start(task, data)

As a direct benefit of the PROCESS_CREATED message, the clone(2) of
processes using pid namespaces now doesn't need to create a pipe
to make the child block until parent setup uid/gid mappings nor it
needs to receive the child pid.

show more ...

Implemented "return" action.

The "return" action can be used to immediately generate a simple HTTP response
with an arbitrary status:

{
"action": {
"return": 404
}
}

This

Implemented "return" action.

The "return" action can be used to immediately generate a simple HTTP response
with an arbitrary status:

{
"action": {
"return": 404
}
}

This is especially useful for denying access to specific resources.

show more ...

Round robin upstream added.

Router: introducing routing on client address.

Isolation: allowed the use of credentials with unpriv userns.

The setuid/setgid syscalls requires root capabilities but if the kernel
supports unprivileged user namespace then the child process has

Isolation: allowed the use of credentials with unpriv userns.

The setuid/setgid syscalls requires root capabilities but if the kernel
supports unprivileged user namespace then the child process has the full
set of capabilities in the new namespace, then we can allow setting "user"
and "group" in such cases (this is a common security use case).

Tests were added to ensure user gets meaningful error messages for
uid/gid mapping misconfigurations.

show more ...