Java: fixed the calculation related to the response buffer.

We need to take into account the size of the nxt_unit_response_t
structure itself when calculating where to start appending data to in
mem

Java: fixed the calculation related to the response buffer.

We need to take into account the size of the nxt_unit_response_t
structure itself when calculating where to start appending data to in
memory.

Closes: <https://github.com/nginx/unit/issues/923>
Reported-by: Alejandro Colomar <alx@kernel.org>
Reviewed-by: Andrew Clayton <a.clayton@nginx.org>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>

show more ...

Fix compilation with GCC and -O0.

Andrei reported an issue with building unit when using '-O0' with GCC
producing the following compiler errors

cc -c -pipe -fPIC -fvisibility=hidden -O -W -Wall -We

Fix compilation with GCC and -O0.

Andrei reported an issue with building unit when using '-O0' with GCC
producing the following compiler errors

cc -c -pipe -fPIC -fvisibility=hidden -O -W -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wmissing-prototypes -Werror -g -O0 -I src -I build \
\
\
-o build/src/nxt_unit.o \
-MMD -MF build/src/nxt_unit.dep -MT build/src/nxt_unit.o \
src/nxt_unit.c
src/nxt_unit.c: In function ‘nxt_unit_log’:
src/nxt_unit.c:6601:9: error: ‘msg’ may be used uninitialized [-Werror=maybe-uninitialized]
6601 | p = nxt_unit_snprint_prefix(p, end, pid, level);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/nxt_unit.c:6682:1: note: by argument 2 of type ‘const char *’ to ‘nxt_unit_snprint_prefix’ declared here
6682 | nxt_unit_snprint_prefix(char *p, const char *end, pid_t pid, int level)
| ^~~~~~~~~~~~~~~~~~~~~~~
src/nxt_unit.c:6582:22: note: ‘msg’ declared here
6582 | char msg[NXT_MAX_ERROR_STR], *p, *end;
| ^~~
src/nxt_unit.c: In function ‘nxt_unit_req_log’:
src/nxt_unit.c:6645:9: error: ‘msg’ may be used uninitialized [-Werror=maybe-uninitialized]
6645 | p = nxt_unit_snprint_prefix(p, end, pid, level);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/nxt_unit.c:6682:1: note: by argument 2 of type ‘const char *’ to ‘nxt_unit_snprint_prefix’ declared here
6682 | nxt_unit_snprint_prefix(char *p, const char *end, pid_t pid, int level)
| ^~~~~~~~~~~~~~~~~~~~~~~
src/nxt_unit.c:6625:35: note: ‘msg’ declared here
6625 | char msg[NXT_MAX_ERROR_STR], *p, *end;
| ^~~
cc1: all warnings being treated as errors

The above was reproduced with

$ ./configure --cc-opt=-O0 && ./configure python && make -j4

This warning doesn't happen on clang (15.0.4) or GCC (8.3) and seems to
have been introduced in GCC 11. The above is from GCC (12.2.1, Fedora
37).

The trigger of this GCC issue is actually part of a commit I introduced
a few months back to constify some function parameters and it seems the
consensus for how to resolve this problem is to simply remove the const
qualifier from the *end parameter to nxt_unit_snprint_prefix().

Reported-by: Andrei Zeliankou <zelenkov@nginx.com>
Link: <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100417>
Link: <https://github.com/samtools/htslib/pull/1285>
Link: <https://gcc.gnu.org/gcc-11/changes.html>
Fixes: 4418f99 ("Constified numerous function parameters.")
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>

show more ...

Added missing error checking in the C API.

pthread_mutex_init(3) may fail for several reasons, and failing to
check will cause Undefined Behavior when those errors happen. Add
missing checks, and c

Added missing error checking in the C API.

pthread_mutex_init(3) may fail for several reasons, and failing to
check will cause Undefined Behavior when those errors happen. Add
missing checks, and correctly deinitialize previously created
stuff before exiting from the API.

Signed-off-by: Alejandro Colomar <alx@nginx.com>
Reviewed-by: Andrew Clayton <a.clayton@nginx.com>
Reviewed-by: Zhidao HONG <z.hong@f5.com>

show more ...

Fixed a mutex leak in the C API.

In nxt_unit_create() we could leak a mutex created in
nxt_unit_ctx_init().

This could happen if nxt_unit_ctx_init() succeeded but later on we
bailed out of nxt_unit

Fixed a mutex leak in the C API.

In nxt_unit_create() we could leak a mutex created in
nxt_unit_ctx_init().

This could happen if nxt_unit_ctx_init() succeeded but later on we
bailed out of nxt_unit_create(), we would destroy the mutex created in
nxt_unit_create() but not the one created in nxt_unit_ctx_init().

Reorder things so that we do the call to nxt_unit_create() after all the
other checks so if it fails we don't leak the mutex it created.

Co-developed-by: Andrew Clayton <a.clayton@f5.com>
Signed-off-by: Andrew Clayton <a.clayton@f5.com>
Signed-off-by: Alex Colomar <a.colomar@f5.com>

show more ...

Unit: removed a useless assignment.

As was pointed out by the cppcheck[0] static code analysis utility there
was a useless assignment in nxt_unit_request_read(). The size parameter
is passed in by v

Unit: removed a useless assignment.

As was pointed out by the cppcheck[0] static code analysis utility there
was a useless assignment in nxt_unit_request_read(). The size parameter
is passed in by value and was being modified without being used again.

[0]: https://cppcheck.sourceforge.io/

show more ...

Unit: avoided needlessly setting lib in nxt_unit_shm_open().

As was pointed out by the cppcheck[0] static code analysis utility, lib
was being set in nxt_unit_shm_open() regardless of platform when

Unit: avoided needlessly setting lib in nxt_unit_shm_open().

As was pointed out by the cppcheck[0] static code analysis utility, lib
was being set in nxt_unit_shm_open() regardless of platform when in fact
it's only used when (NXT_HAVE_MEMFD_CREATE || NXT_HAVE_SHM_OPEN).

Move the variable declaration & definition to be within the

#if (NXT_HAVE_MEMFD_CREATE || NXT_HAVE_SHM_OPEN)

block.

[0]: https://cppcheck.sourceforge.io/

show more ...

Constified numerous function parameters.

As was pointed out by the cppcheck[0] static code analysis utility we
can mark numerous function parameters as 'const'. This acts as a hint to
the compiler a

Constified numerous function parameters.

As was pointed out by the cppcheck[0] static code analysis utility we
can mark numerous function parameters as 'const'. This acts as a hint to
the compiler about our intentions and the compiler will tell us when we
deviate from them.

[0]: https://cppcheck.sourceforge.io/

show more ...

Removing unused tracking fields and functions.

The message tracking is unused since 1d84b9e4b459 commit.

This fixes the issue found by Coverity (CID 376263).

Fixed indentation.

Some lines (incorrectly) had an indentation of 3 or 5, or 7 or 9,
or 11 or 13, or 15 or 17 spaces instead of 4, 8, 12, or 16. Fix them.

Found with:

$ find src -type f | xargs g

Fixed indentation.

Some lines (incorrectly) had an indentation of 3 or 5, or 7 or 9,
or 11 or 13, or 15 or 17 spaces instead of 4, 8, 12, or 16. Fix them.

Found with:

$ find src -type f | xargs grep -n '^ [^ ]';
$ find src -type f | xargs grep -n '^ [^ *]';
$ find src -type f | xargs grep -n '^ [^ ]';
$ find src -type f | xargs grep -n '^ [^ *]';
$ find src -type f | xargs grep -n '^ [^ +]';
$ find src -type f | xargs grep -n '^ [^ *+]';
$ find src -type f | xargs grep -n '^ [^ +]';
$ find src -type f | xargs grep -n '^ [^ *+]';

show more ...

Fixing uninitialized structure field.

Port's "data" field may be used by application and thus need to be set to NULL.
The issue was introduced in the f8a0992944df commit.

Found by Coverity (CID 374

Fixing uninitialized structure field.

Port's "data" field may be used by application and thus need to be set to NULL.
The issue was introduced in the f8a0992944df commit.

Found by Coverity (CID 374352).

show more ...

Sending shared port to application prototype.

Application process started with shared port (and queue) already configured.
But still waits for PORT_ACK message from router to start request processin

Sending shared port to application prototype.

Application process started with shared port (and queue) already configured.
But still waits for PORT_ACK message from router to start request processing
(so-called "ready state").

Waiting for router confirmation is necessary. Otherwise, the application may
produce response and send it to router before the router have the information
about the application process. This is a subject of further optimizations.

show more ...

Fixed possible access to an uninitialized field.

The "recv_msg.incoming_buf" is checked after jumping to the "done" label
if nxt_socket_msg_oob_get_fds() returns an error.

Also moved initialization

Fixed possible access to an uninitialized field.

The "recv_msg.incoming_buf" is checked after jumping to the "done" label
if nxt_socket_msg_oob_get_fds() returns an error.

Also moved initialization of "port_msg" near to its first usage.

Found by Coverity (CID 373899).

show more ...

Introducing application prototype processes.

Introduced SCM_CREDENTIALS / SCM_CREDS in the socket control msgs.

Moving request limit control to libunit.

Introducting application graceful stop. For now only used when application
process reach request limit value.

This closes #585 issue on GitHub.

Fixing warnings on Solaris.

pthread_t on Solaris is an integer type with size not equal to pointer size.
To avoid warnings, type casts to and from pointer needs to be done via
uintptr_t type.

This

Fixing warnings on Solaris.

pthread_t on Solaris is an integer type with size not equal to pointer size.
To avoid warnings, type casts to and from pointer needs to be done via
uintptr_t type.

This change originally proposed by Juraj Lutter <juraj@lutter.sk>.

show more ...

Libunit: processing single port message.

This partially reverts the optimisation introduced in 1d84b9e4b459 to avoid an
unpredictable block in nxt_unit_process_port_msg(). Under high load, this
fun

Libunit: processing single port message.

This partially reverts the optimisation introduced in 1d84b9e4b459 to avoid an
unpredictable block in nxt_unit_process_port_msg(). Under high load, this
function may never return control to its caller, and the external event loop
(in Node.js and Python asyncio) won't be able to process other scheduled
events.

To reproduce the issue, two request processing types are needed: 'fast' and
'furious'. The 'fast' one simply returns a small response, while the 'furious'
schedules asynchronous calls to external resources. Thus, if Unit is subjected
to a large amount of 'fast' requests, the 'furious' request processing freezes
until the high load ends.

The issue was found by Wu Jian Ping (@wujjpp) during Node.js stream
implementation discussion and relates to PR #502 on GitHub.

show more ...

Libunit: fixed shared memory waiting.

The nxt_unit_ctx_port_recv() function may return the NXT_UNIT_AGAIN code, in
which case an attempt to reread the message should be made.

The issue was reproduc

Libunit: fixed shared memory waiting.

The nxt_unit_ctx_port_recv() function may return the NXT_UNIT_AGAIN code, in
which case an attempt to reread the message should be made.

The issue was reproduced in load testing with response sizes 16k and up.
In the rare case of a NXT_UNIT_AGAIN result, a buffer of size -1 was processed,
which triggered a 'message too small' alert; after that, the app process was
terminated.

show more ...

Limiting app queue notifications count in socket.

Under high load, a queue synchonization issue may occur, starting from the
steady state when an app queue message is dequeued immediately after it h

Limiting app queue notifications count in socket.

Under high load, a queue synchonization issue may occur, starting from the
steady state when an app queue message is dequeued immediately after it has been
enqueued. In this state, the router always puts the first message in the queue
and is forced to notify the app about a new message in an empty queue using a
socket pair. On the other hand, the application dequeues and processes the
message without reading the notification from the socket, so the socket buffer
overflows with notifications.

The issue was reproduced during Unit load tests. After a socket buffer
overflow, the router is unable to notify the app about a new first message.
When another message is enqueued, a notification is not required, so the queue
grows without being read by the app. As a result, request processing stops.

This patch changes the notification algorithm by counting the notifications in
the pipe instead of getting the number of messages in the queue.

show more ...

Libunit: improved error logging around initialization env variable.

Libunit: fixing read buffer leakage.

If shared queue is empty, allocated read buffer should be explicitly
released.

Found by Coverity (CID 363943).
The issue was introduced in f5ba5973a0a3.

Libunit: fixing read buffer allocations on exit.

Libunit: closing active requests on quit.

Libunit: making minor tweaks.

Removing unnecessary context operations from shared queue processing loop.
Initializing temporary queues only when required.

Go: removing C proxy functions and re-using goroutines.