Revision tags: 1.32.1-1, 1.32.0-1, 1.32.0, 1.31.1-1, 1.31.1, 1.31.0-1, 1.31.0, 1.30.0-1, 1.30.0 |
|
#
2448:243735980417 |
| 20-Apr-2023 |
Zhidao HONG |
HTTP: added basic URI rewrite.
This commit introduced the basic URI rewrite. It allows users to change request URI. Note the "rewrite" option ignores the contained query if any and the query from th
HTTP: added basic URI rewrite.
This commit introduced the basic URI rewrite. It allows users to change request URI. Note the "rewrite" option ignores the contained query if any and the query from the request is preserverd. An example: "routes": [ { "match": { "uri": "/v1/test" }, "action": { "return": 200 } }, { "action": { "rewrite": "/v1$uri", "pass": "routes" } } ]
Reviewed-by: Alejandro Colomar <alx@nginx.com>
show more ...
|
#
2430:50ce7e6977b6 |
| 31-Mar-2023 |
Alejandro Colomar |
HTTP: optimizing $request_line.
Don't reconstruct a new string for the $request_line from the parsed method, target, and HTTP version, but rather keep a pointer to the original memory where the requ
HTTP: optimizing $request_line.
Don't reconstruct a new string for the $request_line from the parsed method, target, and HTTP version, but rather keep a pointer to the original memory where the request line was received.
This will be necessary for implementing URI rewrites, since we want to log the original request line, and not one constructed from the rewritten target.
This implementation changes behavior (only for invalid requests) in the following way:
Previous behavior was to log as many tokens from the request line as were parsed validly, thus:
Request -> access log ; error log
"GET / HTTP/1.1" -> "GET / HTTP/1.1" OK ; = "GET / HTTP/1.1" -> "GET / HTTP/1.1" [1] ; = "GET / HTTP/2.1" -> "GET / HTTP/2.1" OK ; = "GET / HTTP/1." -> "GET / HTTP/1." [2] ; "GET / HTTP/1. [null]" "GET / food" -> "GET / food" [2] ; "GET / food [null]" "GET / / HTTP/1.1" -> "GET / / HTTP/1.1" [2] ; = "GET / / HTTP/1.1" -> "GET / / HTTP/1.1" [2] ; = "GET food HTTP/1.1" -> "GET" ; "GET [null] [null]" "OPTIONS * HTTP/1.1" -> "OPTIONS" [3] ; "OPTIONS [null] [null]" "FOOBAR baz HTTP/1.1"-> "FOOBAR" ; "FOOBAR [null] [null]" "FOOBAR / HTTP/1.1" -> "FOOBAR / HTTP/1.1" ; = "get / HTTP/1.1" -> "-" ; " [null] [null]" "" -> "-" ; " [null] [null]"
This behavior was rather inconsistent. We have several options to go forward with this patch:
- NGINX behavior.
Log the entire request line, up to '\r' | '\n', even if it was invalid.
This is the most informative alternative. However, RFC-complying requests will probably not send invalid requests.
This information would be interesting to users where debugging requests constructed manually via netcat(1) or a similar tool, or maybe for debugging a client, are important. It might be interesting to support this in the future if our users are interested; for now, since this approach requires looping over invalid requests twice, that's an overhead that we better avoid.
- Previous Unit behavior
This is relatively fast (almost as fast as the next alternative, the one we chose), but the implementation is ugly, in that we need to perform the same operation in many places around the code.
If we want performance, probably the next alternative is better; if we want to be informative, then the first one is better (maybe in combination with the third one too).
- Chosen behavior
Only logging request lines when the request is valid. For any invalid request, or even unsupported ones, the request line will be logged as "-". Thus:
Request -> access log [4]
"GET / HTTP/1.1" -> "GET / HTTP/1.1" OK "GET / HTTP/1.1" -> "GET / HTTP/1.1" [1] "GET / HTTP/2.1" -> "-" [3] "GET / HTTP/1." -> "-" "GET / food" -> "-" "GET / / HTTP/1.1" -> "GET / / HTTP/1.1" [2] "GET / / HTTP/1.1" -> "GET / / HTTP/1.1" [2] "GET food HTTP/1.1" -> "-" "OPTIONS * HTTP/1.1" -> "-" "FOOBAR baz HTTP/1.1"-> "-" "FOOBAR / HTTP/1.1" -> "FOOBAR / HTTP/1.1" "get / HTTP/1.1" -> "-" "" -> "-"
This is less informative than previous behavior, but considering how inconsistent it was, and that RFC-complying agents will probably not send us such requests, we're ready to lose that information in the log. This is of course the fastest and simplest implementation we can get.
We've chosen to implement this alternative in this patch. Since we modified the behavior, this patch also changes the affected tests.
[1]: Multiple successive spaces as a token delimiter is allowed by the RFC, but it is discouraged, and considered a security risk. It is currently supported by Unit, but we will probably drop support for it in the future.
[2]: Unit currently supports spaces in the request-target. This is a violation of the relevant RFC (linked below), and will be fixed in the future, and consider those targets as invalid, returning a 400 (Bad Request), and thus the log lines with the previous inconsistent behavior would be changed.
[3]: Not yet supported.
[4]: In the error log, regarding the "log_routes" conditional logging of the request line, we only need to log the request line if it was valid. It doesn't make sense to log "" or "-" in case that the request was invalid, since this is only useful for understanding decisions of the router. In this case, the access log is more appropriate, which shows that the request was invalid, and a 400 was returned. When the request line is valid, it is printed in the error log exactly as in the access log.
Link: <https://datatracker.ietf.org/doc/html/rfc9112#section-3> Suggested-by: Liam Crilly <liam@nginx.com> Reviewed-by: Zhidao Hong <z.hong@f5.com> Cc: Timo Stark <t.stark@nginx.com> Cc: Andrei Zeliankou <zelenkov@nginx.com> Cc: Andrew Clayton <a.clayton@nginx.com> Cc: Artem Konev <a.konev@f5.com> Signed-off-by: Alejandro Colomar <alx@nginx.com>
show more ...
|
Revision tags: 1.29.1-1, 1.29.1, 1.29.0-1, 1.29.0, 1.28.0-1, 1.28.0 |
|
#
2139:99d792169ffb |
| 16-Jun-2022 |
Andrew Clayton |
Constified numerous function parameters.
As was pointed out by the cppcheck[0] static code analysis utility we can mark numerous function parameters as 'const'. This acts as a hint to the compiler a
Constified numerous function parameters.
As was pointed out by the cppcheck[0] static code analysis utility we can mark numerous function parameters as 'const'. This acts as a hint to the compiler about our intentions and the compiler will tell us when we deviate from them.
[0]: https://cppcheck.sourceforge.io/
show more ...
|
Revision tags: 1.27.0-1, 1.27.0, 1.26.1-1, 1.26.1, 1.26.0-1, 1.26.0, 1.25.0-1, 1.25.0, 1.24.0-1, 1.24.0, 1.23.0-1, 1.23.0, 1.22.0-1, 1.22.0, 1.21.0-1, 1.21.0 |
|
#
1709:1fe93c17d23f |
| 17-Nov-2020 |
Valentin Bartenev |
HTTP parser: allowed more characters in header field names.
Previously, all requests that contained in header field names characters other than alphanumeric, or "-", or "_" were rejected with a 400
HTTP parser: allowed more characters in header field names.
Previously, all requests that contained in header field names characters other than alphanumeric, or "-", or "_" were rejected with a 400 "Bad Request" error response.
Now, the parser allows the same set of characters as specified in RFC 7230, including: "!", "#", "$", "%", "&", "'", "*", "+", ".", "^", "`", "|", and "~". Header field names that contain only these characters are considered valid.
Also, there's a new option introduced: "discard_unsafe_fields". It accepts boolean value and it is set to "true" by default.
When this option is "true", all header field names that contain characters in valid range, but other than alphanumeric or "-" are skipped during parsing. When the option is "false", these header fields aren't skipped.
Requests with non-valid characters in header field names according to RFC 7230 are rejected regardless of "discard_unsafe_fields" setting.
This closes #422 issue on GitHub.
show more ...
|
Revision tags: 1.20.0-1, 1.20.0, 1.19.0-1, 1.19.0 |
|
#
1505:d18f2b38596b |
| 23-Jun-2020 |
Igor Sysoev |
Upstream chunked transfer encoding support.
|
Revision tags: 1.18.0-1, 1.18.0, 1.17.0-1, 1.17.0 |
|
#
1459:358b957ca294 |
| 16-Apr-2020 |
Max Romanov |
Using malloc/free for the http fields hash.
This is required due to lack of a graceful shutdown: there is a small gap between the runtime's memory pool release and router process's exit. Thus, a wor
Using malloc/free for the http fields hash.
This is required due to lack of a graceful shutdown: there is a small gap between the runtime's memory pool release and router process's exit. Thus, a worker thread may start processing a request between these two operations, which may result in an http fields hash access and subsequent crash.
To simplify issue reproduction, it makes sense to add a 2 sec sleep before exit() in nxt_runtime_exit().
show more ...
|
Revision tags: 1.16.0-1, 1.16.0, 1.15.0-1, 1.15.0, 1.14.0-1, 1.14.0, 1.13.0-1, 1.13.0 |
|
#
1270:9efa309be18b |
| 14-Nov-2019 |
Igor Sysoev |
Initial proxy support.
|
Revision tags: 1.12.0-1, 1.12.0 |
|
#
1214:c3666b232602 |
| 30-Sep-2019 |
Valentin Bartenev |
HTTP parser: removed unused "exten" field.
This field was intended for MIME type lookup by file extension when serving static files, but this use case is too narrow; only a fraction of requests targ
HTTP parser: removed unused "exten" field.
This field was intended for MIME type lookup by file extension when serving static files, but this use case is too narrow; only a fraction of requests targets static content, and the URI presumably isn't rewritten. Moreover, current implementation uses the entire filename for MIME type lookup if the file has no extension.
Instead of extracting filenames and extensions when parsing requests, it's easier to obtain them right before serving static content; this behavior is already implemented. Thus, we can drop excessive logic from parser.
show more ...
|
Revision tags: 1.11.0-2, 1.11.0-1, 1.11.0 |
|
#
1170:830d6af7d24c |
| 16-Sep-2019 |
Valentin Bartenev |
HTTP parser: removed unused "plus_in_target" flag.
|
#
1169:f85eea6f1050 |
| 16-Sep-2019 |
Valentin Bartenev |
HTTP parser: removed unused "offset" field.
Thanks to 洪志道 (Hong Zhi Dao).
|
#
1168:d4b329089cca |
| 16-Sep-2019 |
Valentin Bartenev |
HTTP parser: removed unused "exten_start" and "args_start" fields.
|
#
1167:a49ee872e83d |
| 16-Sep-2019 |
Valentin Bartenev |
Configuration: added ability to access object members with slashes.
Now URI encoding can be used to escape "/" in the request path:
GET /config/listeners/unix:%2Fpath%2Fto%2Fsocket/
|
#
1159:7c1765e01a4e |
| 09-Sep-2019 |
Max Romanov |
Added "extern" to nxt_http_fields_hash_proto to avoid link issues.
|
Revision tags: 1.10.0-2, 1.10.0-1, 1.10.0 |
|
#
1126:50a8b6ded8e2 |
| 15-Aug-2019 |
Max Romanov |
Improving response header fields processing.
Fields are filtered one by one before being added to fields list. This avoids adding and then skipping connection-specific fields.
|
Revision tags: 1.9.0-1, 1.9.0 |
|
#
1059:11b44092759d |
| 30-May-2019 |
Igor Sysoev |
Added routing based on header fields.
|
Revision tags: 1.8.0-1, 1.8.0, 1.7.1-1, 1.7.1, 1.7-1, 1.7, 1.6-1, 1.6, 1.5-1, 1.5, 1.4-2, 1.4, 1.3, 1.2, 1.1, 1.0, 0.7, 0.6, 0.5 |
|
#
482:48e1b9d61fbf |
| 15-Jan-2018 |
Valentin Bartenev |
Checking for major HTTP version.
|
#
481:4bfbd8ce0e8a |
| 15-Jan-2018 |
Valentin Bartenev |
Improved HTTP version representation.
|
#
480:11b28d6fc462 |
| 15-Jan-2018 |
Valentin Bartenev |
HTTP parser: improved error reporting.
|
Revision tags: 0.4, 0.3 |
|
#
422:388a61760e7f |
| 27-Dec-2017 |
Valentin Bartenev |
HTTP parser: introduced nxt_http_parse_fields().
|
#
417:47366bb40f2c |
| 25-Dec-2017 |
Valentin Bartenev |
HTTP parser: reworked header fields handling.
|
Revision tags: 0.2, 0.1 |
|
#
112:6f1809436b10 |
| 05-Jul-2017 |
Max Romanov |
Complex target parser copied from NGINX.
nxt_app_request_header_t fields renamed: - 'path' renamed to 'target'. - 'path_no_query' renamed to 'path' and contains parsed value.
|
#
67:5bde485f693f |
| 20-Jun-2017 |
Valentin Bartenev |
HTTP parser: reduced memory consumption of header fields list.
|
#
65:10688b89aa16 |
| 20-Jun-2017 |
Igor Sysoev |
Using new memory pool implementation.
|
#
60:b80bfbd9bddc |
| 13-Jun-2017 |
Valentin Bartenev |
HTTP parser: decoupled header fields processing.
|
#
16:c382e548cbb6 |
| 01-Mar-2017 |
Valentin Bartenev |
HTTP parser.
|