Tests: 8XXX used as default port range.

After the launch of the project, the testing infrastructure was shared with
nginx project in some cases. To avoid port overlap, a decision was made
to shift

Tests: 8XXX used as default port range.

After the launch of the project, the testing infrastructure was shared with
nginx project in some cases. To avoid port overlap, a decision was made
to shift the port range for Unit tests. This problem was resolved a long time
ago and is no longer relevant, so it is now safe to use port 8XXX range as the
default, as it is more appropriate for testing purposes.

show more ...

Tests: get rid of classes in test files.

Class usage came from the unittest framework and it was always redundant
after migration to the pytest. This commit removes classes from files
containing te

Tests: get rid of classes in test files.

Class usage came from the unittest framework and it was always redundant
after migration to the pytest. This commit removes classes from files
containing tests to make them more readable and understandable.

show more ...

Tests: prerequisites checking reworked.

Prerequisites check moved to the module level to simplify class structure.
Discovery and prerequisites checks functions moved to the separate files.
Introduce

Tests: prerequisites checking reworked.

Prerequisites check moved to the module level to simplify class structure.
Discovery and prerequisites checks functions moved to the separate files.
Introduced "require" fixture to provide per-test requirements check.

show more ...

Tests: more fixtures.

Common methods from applications/proto.py converted to the fixtures.
sysctl check moved to the specific file where it is using.
Some options moved to the constructor to have ea

Tests: more fixtures.

Common methods from applications/proto.py converted to the fixtures.
sysctl check moved to the specific file where it is using.
Some options moved to the constructor to have early access.

show more ...

Tests: "temp_dir" fixture used were possible.

Tests: removed unused variables.

HTTP: optimizing $request_line.

Don't reconstruct a new string for the $request_line from the parsed
method, target, and HTTP version, but rather keep a pointer to the
original memory where the requ

HTTP: optimizing $request_line.

Don't reconstruct a new string for the $request_line from the parsed
method, target, and HTTP version, but rather keep a pointer to the
original memory where the request line was received.

This will be necessary for implementing URI rewrites, since we want to
log the original request line, and not one constructed from the
rewritten target.

This implementation changes behavior (only for invalid requests) in the
following way:

Previous behavior was to log as many tokens from the request line as
were parsed validly, thus:

Request -> access log ; error log

"GET / HTTP/1.1" -> "GET / HTTP/1.1" OK ; =
"GET / HTTP/1.1" -> "GET / HTTP/1.1" [1] ; =
"GET / HTTP/2.1" -> "GET / HTTP/2.1" OK ; =
"GET / HTTP/1." -> "GET / HTTP/1." [2] ; "GET / HTTP/1. [null]"
"GET / food" -> "GET / food" [2] ; "GET / food [null]"
"GET / / HTTP/1.1" -> "GET / / HTTP/1.1" [2] ; =
"GET / / HTTP/1.1" -> "GET / / HTTP/1.1" [2] ; =
"GET food HTTP/1.1" -> "GET" ; "GET [null] [null]"
"OPTIONS * HTTP/1.1" -> "OPTIONS" [3] ; "OPTIONS [null] [null]"
"FOOBAR baz HTTP/1.1"-> "FOOBAR" ; "FOOBAR [null] [null]"
"FOOBAR / HTTP/1.1" -> "FOOBAR / HTTP/1.1" ; =
"get / HTTP/1.1" -> "-" ; " [null] [null]"
"" -> "-" ; " [null] [null]"

This behavior was rather inconsistent. We have several options to go
forward with this patch:

- NGINX behavior.

Log the entire request line, up to '\r' | '\n', even if it was
invalid.

This is the most informative alternative. However, RFC-complying
requests will probably not send invalid requests.

This information would be interesting to users where debugging
requests constructed manually via netcat(1) or a similar tool, or
maybe for debugging a client, are important. It might be interesting
to support this in the future if our users are interested; for now,
since this approach requires looping over invalid requests twice,
that's an overhead that we better avoid.

- Previous Unit behavior

This is relatively fast (almost as fast as the next alternative, the
one we chose), but the implementation is ugly, in that we need to
perform the same operation in many places around the code.

If we want performance, probably the next alternative is better; if
we want to be informative, then the first one is better (maybe in
combination with the third one too).

- Chosen behavior

Only logging request lines when the request is valid. For any
invalid request, or even unsupported ones, the request line will be
logged as "-". Thus:

Request -> access log [4]

"GET / HTTP/1.1" -> "GET / HTTP/1.1" OK
"GET / HTTP/1.1" -> "GET / HTTP/1.1" [1]
"GET / HTTP/2.1" -> "-" [3]
"GET / HTTP/1." -> "-"
"GET / food" -> "-"
"GET / / HTTP/1.1" -> "GET / / HTTP/1.1" [2]
"GET / / HTTP/1.1" -> "GET / / HTTP/1.1" [2]
"GET food HTTP/1.1" -> "-"
"OPTIONS * HTTP/1.1" -> "-"
"FOOBAR baz HTTP/1.1"-> "-"
"FOOBAR / HTTP/1.1" -> "FOOBAR / HTTP/1.1"
"get / HTTP/1.1" -> "-"
"" -> "-"

This is less informative than previous behavior, but considering how
inconsistent it was, and that RFC-complying agents will probably not
send us such requests, we're ready to lose that information in the
log. This is of course the fastest and simplest implementation we
can get.

We've chosen to implement this alternative in this patch. Since we
modified the behavior, this patch also changes the affected tests.

[1]: Multiple successive spaces as a token delimiter is allowed by the
RFC, but it is discouraged, and considered a security risk. It is
currently supported by Unit, but we will probably drop support for
it in the future.

[2]: Unit currently supports spaces in the request-target. This is
a violation of the relevant RFC (linked below), and will be fixed
in the future, and consider those targets as invalid, returning
a 400 (Bad Request), and thus the log lines with the previous
inconsistent behavior would be changed.

[3]: Not yet supported.

[4]: In the error log, regarding the "log_routes" conditional logging
of the request line, we only need to log the request line if it
was valid. It doesn't make sense to log "" or "-" in case that
the request was invalid, since this is only useful for
understanding decisions of the router. In this case, the access
log is more appropriate, which shows that the request was invalid,
and a 400 was returned. When the request line is valid, it is
printed in the error log exactly as in the access log.

Link: <https://datatracker.ietf.org/doc/html/rfc9112#section-3>
Suggested-by: Liam Crilly <liam@nginx.com>
Reviewed-by: Zhidao Hong <z.hong@f5.com>
Cc: Timo Stark <t.stark@nginx.com>
Cc: Andrei Zeliankou <zelenkov@nginx.com>
Cc: Andrew Clayton <a.clayton@nginx.com>
Cc: Artem Konev <a.konev@f5.com>
Signed-off-by: Alejandro Colomar <alx@nginx.com>

show more ...

Tests: switched to using f-strings.

Previously, it was necessary to support older versions of Python for
compatibility. F-strings were released in Python 3.6. Python 3.5 was
marked as unsupported

Tests: switched to using f-strings.

Previously, it was necessary to support older versions of Python for
compatibility. F-strings were released in Python 3.6. Python 3.5 was
marked as unsupported by the end of 2020, so now it's possible to start
using f-strings safely for better readability and performance.

show more ...

Tests: reworked "test_variables.py".

Access log used for the variables testing instead of limited routing.
Added missed test for $status variable.
Some tests moved from "test_access_log.py" to "test

Tests: reworked "test_variables.py".

Access log used for the variables testing instead of limited routing.
Added missed test for $status variable.
Some tests moved from "test_access_log.py" to "test_variables.py".

show more ...

Tests: minor fixes.

Tests: added tests for the log format.

Also added tests for the following variables:
$request_line, $time_local, $bytes_sent, and $status.

Tests: style.

Tests: preserving unit.log when run without restart.

Introducing "unit.log.Log" class for "unit.log" file management.
Moving "findall()" function into TestApplicationProto.
Using "os.kill()" to send

Tests: preserving unit.log when run without restart.

Introducing "unit.log.Log" class for "unit.log" file management.
Moving "findall()" function into TestApplicationProto.
Using "os.kill()" to send signals.

show more ...

Tests: style.

Tests: added missing checks for configuration results.

Tests: style.

Tests: unit_stop() removed where possible.

Since wait_for_record() was introduced there is no need
to stop Unit before parsing unit.log.

Tests: options moved to the separate class.

This change is necessary to separate the logic
and prevent possible circular dependency.

Tests: fixed unit.log print.

Tests: minor fixes.

Tests: migrated to the pytest.

Tests: style.

Tests: introduced module version specification in prerequisites.

Tests: redirect tests output to the stdout.

Tests: added USR1 signal test for unit.log.