/unit/docs/ |
H A D | changes.xml | diff 2110:048281cd3d73 Thu May 26 12:38:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Var: Added $request_uri (as in NGINX).
This supports a new variable $request_uri that contains the path and the query (See RFC 3986, section 3). Its contents are percent encoded. This is useful for example to redirect HTTP to HTTPS:
{ "return": "301", "location": "https://$host$request_uri" }
When <http://example.com/foo%23bar?baz> is requested, the server redirects to <https://example.com/foo%23bar?baz>.
===
Testing:
//diff --git a/src/nxt_http_return.c b/src/nxt_http_return.c //index 82c9156..adeb3a1 100644 //--- a/src/nxt_http_return.c //+++ b/src/nxt_http_return.c //@@ -196,6 +196,7 @@ nxt_http_return_send_ready(nxt_task_t *task, void *obj, void *data) // field->value = ctx->encoded.start; // field->value_length = ctx->encoded.length; // } //+ fprintf(stderr, "ALX: target[%1$i]: <%2$.*1$s>\n", (int)r->target.length, r->target.start); // // r->state = &nxt_http_return_send_state; //
{ "listeners": { "*:81": { "pass": "routes/ru" } },
"routes": { "ru": [{ "action": { "return": 301, "location": "$request_uri" } }] } }
$ curl -i http://localhost:81/*foo%2Abar?baz#arg HTTP/1.1 301 Moved Permanently Location: /*foo%2Abar?baz Server: Unit/1.27.0 Date: Mon, 30 May 2022 16:04:30 GMT Content-Length: 0
$ sudo cat /usr/local/unit.log | grep ALX ALX: target[15]: </*foo%2Abar?baz> diff 2081:c68e6afffb84 Tue Apr 05 09:47:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Supporting variables in "location".
............ Description: ............
Before this commit, the encoded URI could be calculated at configuration time. Now, since variables can only be resolved at request time, we have different situations:
- "location" contains no variables:
In this case, we still encode the URI in the conf structure, at configuration time, and then we just copy the resulting string to the ctx structure at request time.
- "location" contains variables:
In this case, we compile the var string at configure time, then when we resolve it at request time, and then we encode the string.
In both cases, as was being done before, if the string is empty, either before or after resolving variables, we skip the encoding.
........... Usefulness: ...........
An example of why this feature may be useful is redirecting HTTP to HTTPS with something like:
"action": { "return": 301, "location": "https://${host}${uri}" }
..... Bugs: .....
This feature conflicts with the relevant RFCs in the following:
'$' is used for Unit variables, but '$' is a reserved character in a URI, to be used as a sub-delimiter. However, it's almost never used as that, and in fact, other parts of Unit already conflict with '$' being a reserved character for use as a sub-delimiter, so this is at least consistent in that sense. VBart suggested an easy workaround if we ever need it: adding a variable '$sign' which resolves to a literal '$'.
...... Notes: ......
An empty string is handled as if "location" wasn't specified at all, so no Location header is sent.
This is incorrect, and the code is slightly misleading.
The Location header consists of a URI-reference[1], which might be a relative one, which itself might consist of an empty string[2].
[1]: <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2> [2]: <https://stackoverflow.com/a/43338457>
Now that we have variables, it's more likely that an empty Location header will be requested, and we should handle it correctly.
I think in a future commit we should modify the code to allow differentiating between an unset "location" and an empty one, which should be treated as any other "location" string.
................. Testing (manual): .................
{ "listeners": { "*:80": { "pass": "routes/str" }, "*:81": { "pass": "routes/empty" }, "*:82": { "pass": "routes/var" }, "*:83": { "pass": "routes/enc-str" }, "*:84": { "pass": "routes/enc-var" } }, "routes": { "str": [ { "action": { "return": 301, "location": "foo" } } ], "empty": [ { "action": { "return": 301, "location": "" } } ], "var": [ { "action": { "return": 301, "location": "$host" } } ], "enc-str": [ { "action": { "return": 301, "location": "f%23o#o" } } ], "enc-var": [ { "action": { "return": 301, "location": "f%23o${host}#o" } } ] } }
$ curl --dump-header - localhost:80 HTTP/1.1 301 Moved Permanently Location: foo Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:06 GMT Content-Length: 0
$ curl --dump-header - localhost:81 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:08 GMT Content-Length: 0
$ curl --dump-header - localhost:82 HTTP/1.1 301 Moved Permanently Location: localhost Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:15 GMT Content-Length: 0
$ curl --dump-header - -H "Host: bar" localhost:82 HTTP/1.1 301 Moved Permanently Location: bar Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:82 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:29 GMT Content-Length: 0
$ curl --dump-header - localhost:83 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:44 GMT Content-Length: 0
$ curl --dump-header - -H "Host: alx" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23oalx#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:52 GMT Content-Length: 0
$ curl --dump-header - -H "Host: a#l%23x" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%2523oa#l%2523x%23o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:23:09 GMT Content-Length: 0
$ curl --dump-header - -H "Host: b##ar" localhost:82 HTTP/1.1 301 Moved Permanently Location: b#%23ar Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:25:01 GMT Content-Length: 0 diff 2081:c68e6afffb84 Tue Apr 05 09:47:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Supporting variables in "location".
............ Description: ............
Before this commit, the encoded URI could be calculated at configuration time. Now, since variables can only be resolved at request time, we have different situations:
- "location" contains no variables:
In this case, we still encode the URI in the conf structure, at configuration time, and then we just copy the resulting string to the ctx structure at request time.
- "location" contains variables:
In this case, we compile the var string at configure time, then when we resolve it at request time, and then we encode the string.
In both cases, as was being done before, if the string is empty, either before or after resolving variables, we skip the encoding.
........... Usefulness: ...........
An example of why this feature may be useful is redirecting HTTP to HTTPS with something like:
"action": { "return": 301, "location": "https://${host}${uri}" }
..... Bugs: .....
This feature conflicts with the relevant RFCs in the following:
'$' is used for Unit variables, but '$' is a reserved character in a URI, to be used as a sub-delimiter. However, it's almost never used as that, and in fact, other parts of Unit already conflict with '$' being a reserved character for use as a sub-delimiter, so this is at least consistent in that sense. VBart suggested an easy workaround if we ever need it: adding a variable '$sign' which resolves to a literal '$'.
...... Notes: ......
An empty string is handled as if "location" wasn't specified at all, so no Location header is sent.
This is incorrect, and the code is slightly misleading.
The Location header consists of a URI-reference[1], which might be a relative one, which itself might consist of an empty string[2].
[1]: <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2> [2]: <https://stackoverflow.com/a/43338457>
Now that we have variables, it's more likely that an empty Location header will be requested, and we should handle it correctly.
I think in a future commit we should modify the code to allow differentiating between an unset "location" and an empty one, which should be treated as any other "location" string.
................. Testing (manual): .................
{ "listeners": { "*:80": { "pass": "routes/str" }, "*:81": { "pass": "routes/empty" }, "*:82": { "pass": "routes/var" }, "*:83": { "pass": "routes/enc-str" }, "*:84": { "pass": "routes/enc-var" } }, "routes": { "str": [ { "action": { "return": 301, "location": "foo" } } ], "empty": [ { "action": { "return": 301, "location": "" } } ], "var": [ { "action": { "return": 301, "location": "$host" } } ], "enc-str": [ { "action": { "return": 301, "location": "f%23o#o" } } ], "enc-var": [ { "action": { "return": 301, "location": "f%23o${host}#o" } } ] } }
$ curl --dump-header - localhost:80 HTTP/1.1 301 Moved Permanently Location: foo Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:06 GMT Content-Length: 0
$ curl --dump-header - localhost:81 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:08 GMT Content-Length: 0
$ curl --dump-header - localhost:82 HTTP/1.1 301 Moved Permanently Location: localhost Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:15 GMT Content-Length: 0
$ curl --dump-header - -H "Host: bar" localhost:82 HTTP/1.1 301 Moved Permanently Location: bar Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:82 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:29 GMT Content-Length: 0
$ curl --dump-header - localhost:83 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:44 GMT Content-Length: 0
$ curl --dump-header - -H "Host: alx" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23oalx#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:52 GMT Content-Length: 0
$ curl --dump-header - -H "Host: a#l%23x" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%2523oa#l%2523x%23o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:23:09 GMT Content-Length: 0
$ curl --dump-header - -H "Host: b##ar" localhost:82 HTTP/1.1 301 Moved Permanently Location: b#%23ar Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:25:01 GMT Content-Length: 0 diff 2081:c68e6afffb84 Tue Apr 05 09:47:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Supporting variables in "location".
............ Description: ............
Before this commit, the encoded URI could be calculated at configuration time. Now, since variables can only be resolved at request time, we have different situations:
- "location" contains no variables:
In this case, we still encode the URI in the conf structure, at configuration time, and then we just copy the resulting string to the ctx structure at request time.
- "location" contains variables:
In this case, we compile the var string at configure time, then when we resolve it at request time, and then we encode the string.
In both cases, as was being done before, if the string is empty, either before or after resolving variables, we skip the encoding.
........... Usefulness: ...........
An example of why this feature may be useful is redirecting HTTP to HTTPS with something like:
"action": { "return": 301, "location": "https://${host}${uri}" }
..... Bugs: .....
This feature conflicts with the relevant RFCs in the following:
'$' is used for Unit variables, but '$' is a reserved character in a URI, to be used as a sub-delimiter. However, it's almost never used as that, and in fact, other parts of Unit already conflict with '$' being a reserved character for use as a sub-delimiter, so this is at least consistent in that sense. VBart suggested an easy workaround if we ever need it: adding a variable '$sign' which resolves to a literal '$'.
...... Notes: ......
An empty string is handled as if "location" wasn't specified at all, so no Location header is sent.
This is incorrect, and the code is slightly misleading.
The Location header consists of a URI-reference[1], which might be a relative one, which itself might consist of an empty string[2].
[1]: <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2> [2]: <https://stackoverflow.com/a/43338457>
Now that we have variables, it's more likely that an empty Location header will be requested, and we should handle it correctly.
I think in a future commit we should modify the code to allow differentiating between an unset "location" and an empty one, which should be treated as any other "location" string.
................. Testing (manual): .................
{ "listeners": { "*:80": { "pass": "routes/str" }, "*:81": { "pass": "routes/empty" }, "*:82": { "pass": "routes/var" }, "*:83": { "pass": "routes/enc-str" }, "*:84": { "pass": "routes/enc-var" } }, "routes": { "str": [ { "action": { "return": 301, "location": "foo" } } ], "empty": [ { "action": { "return": 301, "location": "" } } ], "var": [ { "action": { "return": 301, "location": "$host" } } ], "enc-str": [ { "action": { "return": 301, "location": "f%23o#o" } } ], "enc-var": [ { "action": { "return": 301, "location": "f%23o${host}#o" } } ] } }
$ curl --dump-header - localhost:80 HTTP/1.1 301 Moved Permanently Location: foo Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:06 GMT Content-Length: 0
$ curl --dump-header - localhost:81 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:08 GMT Content-Length: 0
$ curl --dump-header - localhost:82 HTTP/1.1 301 Moved Permanently Location: localhost Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:15 GMT Content-Length: 0
$ curl --dump-header - -H "Host: bar" localhost:82 HTTP/1.1 301 Moved Permanently Location: bar Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:82 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:29 GMT Content-Length: 0
$ curl --dump-header - localhost:83 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:44 GMT Content-Length: 0
$ curl --dump-header - -H "Host: alx" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23oalx#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:52 GMT Content-Length: 0
$ curl --dump-header - -H "Host: a#l%23x" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%2523oa#l%2523x%23o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:23:09 GMT Content-Length: 0
$ curl --dump-header - -H "Host: b##ar" localhost:82 HTTP/1.1 301 Moved Permanently Location: b#%23ar Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:25:01 GMT Content-Length: 0 diff 2081:c68e6afffb84 Tue Apr 05 09:47:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Supporting variables in "location".
............ Description: ............
Before this commit, the encoded URI could be calculated at configuration time. Now, since variables can only be resolved at request time, we have different situations:
- "location" contains no variables:
In this case, we still encode the URI in the conf structure, at configuration time, and then we just copy the resulting string to the ctx structure at request time.
- "location" contains variables:
In this case, we compile the var string at configure time, then when we resolve it at request time, and then we encode the string.
In both cases, as was being done before, if the string is empty, either before or after resolving variables, we skip the encoding.
........... Usefulness: ...........
An example of why this feature may be useful is redirecting HTTP to HTTPS with something like:
"action": { "return": 301, "location": "https://${host}${uri}" }
..... Bugs: .....
This feature conflicts with the relevant RFCs in the following:
'$' is used for Unit variables, but '$' is a reserved character in a URI, to be used as a sub-delimiter. However, it's almost never used as that, and in fact, other parts of Unit already conflict with '$' being a reserved character for use as a sub-delimiter, so this is at least consistent in that sense. VBart suggested an easy workaround if we ever need it: adding a variable '$sign' which resolves to a literal '$'.
...... Notes: ......
An empty string is handled as if "location" wasn't specified at all, so no Location header is sent.
This is incorrect, and the code is slightly misleading.
The Location header consists of a URI-reference[1], which might be a relative one, which itself might consist of an empty string[2].
[1]: <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2> [2]: <https://stackoverflow.com/a/43338457>
Now that we have variables, it's more likely that an empty Location header will be requested, and we should handle it correctly.
I think in a future commit we should modify the code to allow differentiating between an unset "location" and an empty one, which should be treated as any other "location" string.
................. Testing (manual): .................
{ "listeners": { "*:80": { "pass": "routes/str" }, "*:81": { "pass": "routes/empty" }, "*:82": { "pass": "routes/var" }, "*:83": { "pass": "routes/enc-str" }, "*:84": { "pass": "routes/enc-var" } }, "routes": { "str": [ { "action": { "return": 301, "location": "foo" } } ], "empty": [ { "action": { "return": 301, "location": "" } } ], "var": [ { "action": { "return": 301, "location": "$host" } } ], "enc-str": [ { "action": { "return": 301, "location": "f%23o#o" } } ], "enc-var": [ { "action": { "return": 301, "location": "f%23o${host}#o" } } ] } }
$ curl --dump-header - localhost:80 HTTP/1.1 301 Moved Permanently Location: foo Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:06 GMT Content-Length: 0
$ curl --dump-header - localhost:81 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:08 GMT Content-Length: 0
$ curl --dump-header - localhost:82 HTTP/1.1 301 Moved Permanently Location: localhost Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:15 GMT Content-Length: 0
$ curl --dump-header - -H "Host: bar" localhost:82 HTTP/1.1 301 Moved Permanently Location: bar Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:82 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:29 GMT Content-Length: 0
$ curl --dump-header - localhost:83 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:44 GMT Content-Length: 0
$ curl --dump-header - -H "Host: alx" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23oalx#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:52 GMT Content-Length: 0
$ curl --dump-header - -H "Host: a#l%23x" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%2523oa#l%2523x%23o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:23:09 GMT Content-Length: 0
$ curl --dump-header - -H "Host: b##ar" localhost:82 HTTP/1.1 301 Moved Permanently Location: b#%23ar Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:25:01 GMT Content-Length: 0 diff 2081:c68e6afffb84 Tue Apr 05 09:47:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Supporting variables in "location".
............ Description: ............
Before this commit, the encoded URI could be calculated at configuration time. Now, since variables can only be resolved at request time, we have different situations:
- "location" contains no variables:
In this case, we still encode the URI in the conf structure, at configuration time, and then we just copy the resulting string to the ctx structure at request time.
- "location" contains variables:
In this case, we compile the var string at configure time, then when we resolve it at request time, and then we encode the string.
In both cases, as was being done before, if the string is empty, either before or after resolving variables, we skip the encoding.
........... Usefulness: ...........
An example of why this feature may be useful is redirecting HTTP to HTTPS with something like:
"action": { "return": 301, "location": "https://${host}${uri}" }
..... Bugs: .....
This feature conflicts with the relevant RFCs in the following:
'$' is used for Unit variables, but '$' is a reserved character in a URI, to be used as a sub-delimiter. However, it's almost never used as that, and in fact, other parts of Unit already conflict with '$' being a reserved character for use as a sub-delimiter, so this is at least consistent in that sense. VBart suggested an easy workaround if we ever need it: adding a variable '$sign' which resolves to a literal '$'.
...... Notes: ......
An empty string is handled as if "location" wasn't specified at all, so no Location header is sent.
This is incorrect, and the code is slightly misleading.
The Location header consists of a URI-reference[1], which might be a relative one, which itself might consist of an empty string[2].
[1]: <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2> [2]: <https://stackoverflow.com/a/43338457>
Now that we have variables, it's more likely that an empty Location header will be requested, and we should handle it correctly.
I think in a future commit we should modify the code to allow differentiating between an unset "location" and an empty one, which should be treated as any other "location" string.
................. Testing (manual): .................
{ "listeners": { "*:80": { "pass": "routes/str" }, "*:81": { "pass": "routes/empty" }, "*:82": { "pass": "routes/var" }, "*:83": { "pass": "routes/enc-str" }, "*:84": { "pass": "routes/enc-var" } }, "routes": { "str": [ { "action": { "return": 301, "location": "foo" } } ], "empty": [ { "action": { "return": 301, "location": "" } } ], "var": [ { "action": { "return": 301, "location": "$host" } } ], "enc-str": [ { "action": { "return": 301, "location": "f%23o#o" } } ], "enc-var": [ { "action": { "return": 301, "location": "f%23o${host}#o" } } ] } }
$ curl --dump-header - localhost:80 HTTP/1.1 301 Moved Permanently Location: foo Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:06 GMT Content-Length: 0
$ curl --dump-header - localhost:81 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:08 GMT Content-Length: 0
$ curl --dump-header - localhost:82 HTTP/1.1 301 Moved Permanently Location: localhost Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:15 GMT Content-Length: 0
$ curl --dump-header - -H "Host: bar" localhost:82 HTTP/1.1 301 Moved Permanently Location: bar Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:82 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:29 GMT Content-Length: 0
$ curl --dump-header - localhost:83 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:44 GMT Content-Length: 0
$ curl --dump-header - -H "Host: alx" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23oalx#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:52 GMT Content-Length: 0
$ curl --dump-header - -H "Host: a#l%23x" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%2523oa#l%2523x%23o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:23:09 GMT Content-Length: 0
$ curl --dump-header - -H "Host: b##ar" localhost:82 HTTP/1.1 301 Moved Permanently Location: b#%23ar Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:25:01 GMT Content-Length: 0
|
/unit/src/ |
H A D | nxt_conf_validation.c | diff 2081:c68e6afffb84 Tue Apr 05 09:47:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Supporting variables in "location".
............ Description: ............
Before this commit, the encoded URI could be calculated at configuration time. Now, since variables can only be resolved at request time, we have different situations:
- "location" contains no variables:
In this case, we still encode the URI in the conf structure, at configuration time, and then we just copy the resulting string to the ctx structure at request time.
- "location" contains variables:
In this case, we compile the var string at configure time, then when we resolve it at request time, and then we encode the string.
In both cases, as was being done before, if the string is empty, either before or after resolving variables, we skip the encoding.
........... Usefulness: ...........
An example of why this feature may be useful is redirecting HTTP to HTTPS with something like:
"action": { "return": 301, "location": "https://${host}${uri}" }
..... Bugs: .....
This feature conflicts with the relevant RFCs in the following:
'$' is used for Unit variables, but '$' is a reserved character in a URI, to be used as a sub-delimiter. However, it's almost never used as that, and in fact, other parts of Unit already conflict with '$' being a reserved character for use as a sub-delimiter, so this is at least consistent in that sense. VBart suggested an easy workaround if we ever need it: adding a variable '$sign' which resolves to a literal '$'.
...... Notes: ......
An empty string is handled as if "location" wasn't specified at all, so no Location header is sent.
This is incorrect, and the code is slightly misleading.
The Location header consists of a URI-reference[1], which might be a relative one, which itself might consist of an empty string[2].
[1]: <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2> [2]: <https://stackoverflow.com/a/43338457>
Now that we have variables, it's more likely that an empty Location header will be requested, and we should handle it correctly.
I think in a future commit we should modify the code to allow differentiating between an unset "location" and an empty one, which should be treated as any other "location" string.
................. Testing (manual): .................
{ "listeners": { "*:80": { "pass": "routes/str" }, "*:81": { "pass": "routes/empty" }, "*:82": { "pass": "routes/var" }, "*:83": { "pass": "routes/enc-str" }, "*:84": { "pass": "routes/enc-var" } }, "routes": { "str": [ { "action": { "return": 301, "location": "foo" } } ], "empty": [ { "action": { "return": 301, "location": "" } } ], "var": [ { "action": { "return": 301, "location": "$host" } } ], "enc-str": [ { "action": { "return": 301, "location": "f%23o#o" } } ], "enc-var": [ { "action": { "return": 301, "location": "f%23o${host}#o" } } ] } }
$ curl --dump-header - localhost:80 HTTP/1.1 301 Moved Permanently Location: foo Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:06 GMT Content-Length: 0
$ curl --dump-header - localhost:81 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:08 GMT Content-Length: 0
$ curl --dump-header - localhost:82 HTTP/1.1 301 Moved Permanently Location: localhost Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:15 GMT Content-Length: 0
$ curl --dump-header - -H "Host: bar" localhost:82 HTTP/1.1 301 Moved Permanently Location: bar Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:82 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:29 GMT Content-Length: 0
$ curl --dump-header - localhost:83 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:44 GMT Content-Length: 0
$ curl --dump-header - -H "Host: alx" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23oalx#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:52 GMT Content-Length: 0
$ curl --dump-header - -H "Host: a#l%23x" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%2523oa#l%2523x%23o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:23:09 GMT Content-Length: 0
$ curl --dump-header - -H "Host: b##ar" localhost:82 HTTP/1.1 301 Moved Permanently Location: b#%23ar Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:25:01 GMT Content-Length: 0 diff 2081:c68e6afffb84 Tue Apr 05 09:47:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Supporting variables in "location".
............ Description: ............
Before this commit, the encoded URI could be calculated at configuration time. Now, since variables can only be resolved at request time, we have different situations:
- "location" contains no variables:
In this case, we still encode the URI in the conf structure, at configuration time, and then we just copy the resulting string to the ctx structure at request time.
- "location" contains variables:
In this case, we compile the var string at configure time, then when we resolve it at request time, and then we encode the string.
In both cases, as was being done before, if the string is empty, either before or after resolving variables, we skip the encoding.
........... Usefulness: ...........
An example of why this feature may be useful is redirecting HTTP to HTTPS with something like:
"action": { "return": 301, "location": "https://${host}${uri}" }
..... Bugs: .....
This feature conflicts with the relevant RFCs in the following:
'$' is used for Unit variables, but '$' is a reserved character in a URI, to be used as a sub-delimiter. However, it's almost never used as that, and in fact, other parts of Unit already conflict with '$' being a reserved character for use as a sub-delimiter, so this is at least consistent in that sense. VBart suggested an easy workaround if we ever need it: adding a variable '$sign' which resolves to a literal '$'.
...... Notes: ......
An empty string is handled as if "location" wasn't specified at all, so no Location header is sent.
This is incorrect, and the code is slightly misleading.
The Location header consists of a URI-reference[1], which might be a relative one, which itself might consist of an empty string[2].
[1]: <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2> [2]: <https://stackoverflow.com/a/43338457>
Now that we have variables, it's more likely that an empty Location header will be requested, and we should handle it correctly.
I think in a future commit we should modify the code to allow differentiating between an unset "location" and an empty one, which should be treated as any other "location" string.
................. Testing (manual): .................
{ "listeners": { "*:80": { "pass": "routes/str" }, "*:81": { "pass": "routes/empty" }, "*:82": { "pass": "routes/var" }, "*:83": { "pass": "routes/enc-str" }, "*:84": { "pass": "routes/enc-var" } }, "routes": { "str": [ { "action": { "return": 301, "location": "foo" } } ], "empty": [ { "action": { "return": 301, "location": "" } } ], "var": [ { "action": { "return": 301, "location": "$host" } } ], "enc-str": [ { "action": { "return": 301, "location": "f%23o#o" } } ], "enc-var": [ { "action": { "return": 301, "location": "f%23o${host}#o" } } ] } }
$ curl --dump-header - localhost:80 HTTP/1.1 301 Moved Permanently Location: foo Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:06 GMT Content-Length: 0
$ curl --dump-header - localhost:81 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:08 GMT Content-Length: 0
$ curl --dump-header - localhost:82 HTTP/1.1 301 Moved Permanently Location: localhost Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:15 GMT Content-Length: 0
$ curl --dump-header - -H "Host: bar" localhost:82 HTTP/1.1 301 Moved Permanently Location: bar Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:82 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:29 GMT Content-Length: 0
$ curl --dump-header - localhost:83 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:44 GMT Content-Length: 0
$ curl --dump-header - -H "Host: alx" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23oalx#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:52 GMT Content-Length: 0
$ curl --dump-header - -H "Host: a#l%23x" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%2523oa#l%2523x%23o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:23:09 GMT Content-Length: 0
$ curl --dump-header - -H "Host: b##ar" localhost:82 HTTP/1.1 301 Moved Permanently Location: b#%23ar Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:25:01 GMT Content-Length: 0 diff 2081:c68e6afffb84 Tue Apr 05 09:47:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Supporting variables in "location".
............ Description: ............
Before this commit, the encoded URI could be calculated at configuration time. Now, since variables can only be resolved at request time, we have different situations:
- "location" contains no variables:
In this case, we still encode the URI in the conf structure, at configuration time, and then we just copy the resulting string to the ctx structure at request time.
- "location" contains variables:
In this case, we compile the var string at configure time, then when we resolve it at request time, and then we encode the string.
In both cases, as was being done before, if the string is empty, either before or after resolving variables, we skip the encoding.
........... Usefulness: ...........
An example of why this feature may be useful is redirecting HTTP to HTTPS with something like:
"action": { "return": 301, "location": "https://${host}${uri}" }
..... Bugs: .....
This feature conflicts with the relevant RFCs in the following:
'$' is used for Unit variables, but '$' is a reserved character in a URI, to be used as a sub-delimiter. However, it's almost never used as that, and in fact, other parts of Unit already conflict with '$' being a reserved character for use as a sub-delimiter, so this is at least consistent in that sense. VBart suggested an easy workaround if we ever need it: adding a variable '$sign' which resolves to a literal '$'.
...... Notes: ......
An empty string is handled as if "location" wasn't specified at all, so no Location header is sent.
This is incorrect, and the code is slightly misleading.
The Location header consists of a URI-reference[1], which might be a relative one, which itself might consist of an empty string[2].
[1]: <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2> [2]: <https://stackoverflow.com/a/43338457>
Now that we have variables, it's more likely that an empty Location header will be requested, and we should handle it correctly.
I think in a future commit we should modify the code to allow differentiating between an unset "location" and an empty one, which should be treated as any other "location" string.
................. Testing (manual): .................
{ "listeners": { "*:80": { "pass": "routes/str" }, "*:81": { "pass": "routes/empty" }, "*:82": { "pass": "routes/var" }, "*:83": { "pass": "routes/enc-str" }, "*:84": { "pass": "routes/enc-var" } }, "routes": { "str": [ { "action": { "return": 301, "location": "foo" } } ], "empty": [ { "action": { "return": 301, "location": "" } } ], "var": [ { "action": { "return": 301, "location": "$host" } } ], "enc-str": [ { "action": { "return": 301, "location": "f%23o#o" } } ], "enc-var": [ { "action": { "return": 301, "location": "f%23o${host}#o" } } ] } }
$ curl --dump-header - localhost:80 HTTP/1.1 301 Moved Permanently Location: foo Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:06 GMT Content-Length: 0
$ curl --dump-header - localhost:81 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:08 GMT Content-Length: 0
$ curl --dump-header - localhost:82 HTTP/1.1 301 Moved Permanently Location: localhost Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:15 GMT Content-Length: 0
$ curl --dump-header - -H "Host: bar" localhost:82 HTTP/1.1 301 Moved Permanently Location: bar Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:82 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:29 GMT Content-Length: 0
$ curl --dump-header - localhost:83 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:44 GMT Content-Length: 0
$ curl --dump-header - -H "Host: alx" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23oalx#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:52 GMT Content-Length: 0
$ curl --dump-header - -H "Host: a#l%23x" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%2523oa#l%2523x%23o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:23:09 GMT Content-Length: 0
$ curl --dump-header - -H "Host: b##ar" localhost:82 HTTP/1.1 301 Moved Permanently Location: b#%23ar Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:25:01 GMT Content-Length: 0 diff 2081:c68e6afffb84 Tue Apr 05 09:47:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Supporting variables in "location".
............ Description: ............
Before this commit, the encoded URI could be calculated at configuration time. Now, since variables can only be resolved at request time, we have different situations:
- "location" contains no variables:
In this case, we still encode the URI in the conf structure, at configuration time, and then we just copy the resulting string to the ctx structure at request time.
- "location" contains variables:
In this case, we compile the var string at configure time, then when we resolve it at request time, and then we encode the string.
In both cases, as was being done before, if the string is empty, either before or after resolving variables, we skip the encoding.
........... Usefulness: ...........
An example of why this feature may be useful is redirecting HTTP to HTTPS with something like:
"action": { "return": 301, "location": "https://${host}${uri}" }
..... Bugs: .....
This feature conflicts with the relevant RFCs in the following:
'$' is used for Unit variables, but '$' is a reserved character in a URI, to be used as a sub-delimiter. However, it's almost never used as that, and in fact, other parts of Unit already conflict with '$' being a reserved character for use as a sub-delimiter, so this is at least consistent in that sense. VBart suggested an easy workaround if we ever need it: adding a variable '$sign' which resolves to a literal '$'.
...... Notes: ......
An empty string is handled as if "location" wasn't specified at all, so no Location header is sent.
This is incorrect, and the code is slightly misleading.
The Location header consists of a URI-reference[1], which might be a relative one, which itself might consist of an empty string[2].
[1]: <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2> [2]: <https://stackoverflow.com/a/43338457>
Now that we have variables, it's more likely that an empty Location header will be requested, and we should handle it correctly.
I think in a future commit we should modify the code to allow differentiating between an unset "location" and an empty one, which should be treated as any other "location" string.
................. Testing (manual): .................
{ "listeners": { "*:80": { "pass": "routes/str" }, "*:81": { "pass": "routes/empty" }, "*:82": { "pass": "routes/var" }, "*:83": { "pass": "routes/enc-str" }, "*:84": { "pass": "routes/enc-var" } }, "routes": { "str": [ { "action": { "return": 301, "location": "foo" } } ], "empty": [ { "action": { "return": 301, "location": "" } } ], "var": [ { "action": { "return": 301, "location": "$host" } } ], "enc-str": [ { "action": { "return": 301, "location": "f%23o#o" } } ], "enc-var": [ { "action": { "return": 301, "location": "f%23o${host}#o" } } ] } }
$ curl --dump-header - localhost:80 HTTP/1.1 301 Moved Permanently Location: foo Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:06 GMT Content-Length: 0
$ curl --dump-header - localhost:81 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:08 GMT Content-Length: 0
$ curl --dump-header - localhost:82 HTTP/1.1 301 Moved Permanently Location: localhost Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:15 GMT Content-Length: 0
$ curl --dump-header - -H "Host: bar" localhost:82 HTTP/1.1 301 Moved Permanently Location: bar Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:82 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:29 GMT Content-Length: 0
$ curl --dump-header - localhost:83 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:44 GMT Content-Length: 0
$ curl --dump-header - -H "Host: alx" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23oalx#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:52 GMT Content-Length: 0
$ curl --dump-header - -H "Host: a#l%23x" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%2523oa#l%2523x%23o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:23:09 GMT Content-Length: 0
$ curl --dump-header - -H "Host: b##ar" localhost:82 HTTP/1.1 301 Moved Permanently Location: b#%23ar Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:25:01 GMT Content-Length: 0 diff 2081:c68e6afffb84 Tue Apr 05 09:47:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Supporting variables in "location".
............ Description: ............
Before this commit, the encoded URI could be calculated at configuration time. Now, since variables can only be resolved at request time, we have different situations:
- "location" contains no variables:
In this case, we still encode the URI in the conf structure, at configuration time, and then we just copy the resulting string to the ctx structure at request time.
- "location" contains variables:
In this case, we compile the var string at configure time, then when we resolve it at request time, and then we encode the string.
In both cases, as was being done before, if the string is empty, either before or after resolving variables, we skip the encoding.
........... Usefulness: ...........
An example of why this feature may be useful is redirecting HTTP to HTTPS with something like:
"action": { "return": 301, "location": "https://${host}${uri}" }
..... Bugs: .....
This feature conflicts with the relevant RFCs in the following:
'$' is used for Unit variables, but '$' is a reserved character in a URI, to be used as a sub-delimiter. However, it's almost never used as that, and in fact, other parts of Unit already conflict with '$' being a reserved character for use as a sub-delimiter, so this is at least consistent in that sense. VBart suggested an easy workaround if we ever need it: adding a variable '$sign' which resolves to a literal '$'.
...... Notes: ......
An empty string is handled as if "location" wasn't specified at all, so no Location header is sent.
This is incorrect, and the code is slightly misleading.
The Location header consists of a URI-reference[1], which might be a relative one, which itself might consist of an empty string[2].
[1]: <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2> [2]: <https://stackoverflow.com/a/43338457>
Now that we have variables, it's more likely that an empty Location header will be requested, and we should handle it correctly.
I think in a future commit we should modify the code to allow differentiating between an unset "location" and an empty one, which should be treated as any other "location" string.
................. Testing (manual): .................
{ "listeners": { "*:80": { "pass": "routes/str" }, "*:81": { "pass": "routes/empty" }, "*:82": { "pass": "routes/var" }, "*:83": { "pass": "routes/enc-str" }, "*:84": { "pass": "routes/enc-var" } }, "routes": { "str": [ { "action": { "return": 301, "location": "foo" } } ], "empty": [ { "action": { "return": 301, "location": "" } } ], "var": [ { "action": { "return": 301, "location": "$host" } } ], "enc-str": [ { "action": { "return": 301, "location": "f%23o#o" } } ], "enc-var": [ { "action": { "return": 301, "location": "f%23o${host}#o" } } ] } }
$ curl --dump-header - localhost:80 HTTP/1.1 301 Moved Permanently Location: foo Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:06 GMT Content-Length: 0
$ curl --dump-header - localhost:81 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:08 GMT Content-Length: 0
$ curl --dump-header - localhost:82 HTTP/1.1 301 Moved Permanently Location: localhost Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:15 GMT Content-Length: 0
$ curl --dump-header - -H "Host: bar" localhost:82 HTTP/1.1 301 Moved Permanently Location: bar Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:82 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:29 GMT Content-Length: 0
$ curl --dump-header - localhost:83 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:44 GMT Content-Length: 0
$ curl --dump-header - -H "Host: alx" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23oalx#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:52 GMT Content-Length: 0
$ curl --dump-header - -H "Host: a#l%23x" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%2523oa#l%2523x%23o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:23:09 GMT Content-Length: 0
$ curl --dump-header - -H "Host: b##ar" localhost:82 HTTP/1.1 301 Moved Permanently Location: b#%23ar Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:25:01 GMT Content-Length: 0
|
H A D | nxt_controller.c | diff 342:82c2825a617a Wed Oct 04 11:58:00 UTC 2017 Max Romanov <max.romanov@nginx.com> Using engine memiory pool for port write allocations.
To allow use port from different threads, the first step is to avoid using port's memory pool for temporary allocations required to send data through the port. Including but not limited by: - buffers for data; - send message structures; - new mmap fd notifications;
It is still safe to use port memory pool for incoming buffers allocations because recieve operation bound to single thread.
|
H A D | nxt_http_parse.c | diff 574:1ec560778942 Thu Mar 15 17:52:00 UTC 2018 Valentin Bartenev <vbart@nginx.com> HTTP parser: fixed parsing of field values ending with space.
This closes #82 issue on GitHub. diff 454:82c827d1cd13 Tue Jan 09 13:50:00 UTC 2018 Valentin Bartenev <vbart@nginx.com> HTTP parser: allowing underscore in header field names.
|
H A D | nxt_http_return.c | diff 2081:c68e6afffb84 Tue Apr 05 09:47:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Supporting variables in "location".
............ Description: ............
Before this commit, the encoded URI could be calculated at configuration time. Now, since variables can only be resolved at request time, we have different situations:
- "location" contains no variables:
In this case, we still encode the URI in the conf structure, at configuration time, and then we just copy the resulting string to the ctx structure at request time.
- "location" contains variables:
In this case, we compile the var string at configure time, then when we resolve it at request time, and then we encode the string.
In both cases, as was being done before, if the string is empty, either before or after resolving variables, we skip the encoding.
........... Usefulness: ...........
An example of why this feature may be useful is redirecting HTTP to HTTPS with something like:
"action": { "return": 301, "location": "https://${host}${uri}" }
..... Bugs: .....
This feature conflicts with the relevant RFCs in the following:
'$' is used for Unit variables, but '$' is a reserved character in a URI, to be used as a sub-delimiter. However, it's almost never used as that, and in fact, other parts of Unit already conflict with '$' being a reserved character for use as a sub-delimiter, so this is at least consistent in that sense. VBart suggested an easy workaround if we ever need it: adding a variable '$sign' which resolves to a literal '$'.
...... Notes: ......
An empty string is handled as if "location" wasn't specified at all, so no Location header is sent.
This is incorrect, and the code is slightly misleading.
The Location header consists of a URI-reference[1], which might be a relative one, which itself might consist of an empty string[2].
[1]: <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2> [2]: <https://stackoverflow.com/a/43338457>
Now that we have variables, it's more likely that an empty Location header will be requested, and we should handle it correctly.
I think in a future commit we should modify the code to allow differentiating between an unset "location" and an empty one, which should be treated as any other "location" string.
................. Testing (manual): .................
{ "listeners": { "*:80": { "pass": "routes/str" }, "*:81": { "pass": "routes/empty" }, "*:82": { "pass": "routes/var" }, "*:83": { "pass": "routes/enc-str" }, "*:84": { "pass": "routes/enc-var" } }, "routes": { "str": [ { "action": { "return": 301, "location": "foo" } } ], "empty": [ { "action": { "return": 301, "location": "" } } ], "var": [ { "action": { "return": 301, "location": "$host" } } ], "enc-str": [ { "action": { "return": 301, "location": "f%23o#o" } } ], "enc-var": [ { "action": { "return": 301, "location": "f%23o${host}#o" } } ] } }
$ curl --dump-header - localhost:80 HTTP/1.1 301 Moved Permanently Location: foo Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:06 GMT Content-Length: 0
$ curl --dump-header - localhost:81 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:08 GMT Content-Length: 0
$ curl --dump-header - localhost:82 HTTP/1.1 301 Moved Permanently Location: localhost Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:15 GMT Content-Length: 0
$ curl --dump-header - -H "Host: bar" localhost:82 HTTP/1.1 301 Moved Permanently Location: bar Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:82 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:29 GMT Content-Length: 0
$ curl --dump-header - localhost:83 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:44 GMT Content-Length: 0
$ curl --dump-header - -H "Host: alx" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23oalx#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:52 GMT Content-Length: 0
$ curl --dump-header - -H "Host: a#l%23x" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%2523oa#l%2523x%23o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:23:09 GMT Content-Length: 0
$ curl --dump-header - -H "Host: b##ar" localhost:82 HTTP/1.1 301 Moved Permanently Location: b#%23ar Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:25:01 GMT Content-Length: 0 diff 2081:c68e6afffb84 Tue Apr 05 09:47:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Supporting variables in "location".
............ Description: ............
Before this commit, the encoded URI could be calculated at configuration time. Now, since variables can only be resolved at request time, we have different situations:
- "location" contains no variables:
In this case, we still encode the URI in the conf structure, at configuration time, and then we just copy the resulting string to the ctx structure at request time.
- "location" contains variables:
In this case, we compile the var string at configure time, then when we resolve it at request time, and then we encode the string.
In both cases, as was being done before, if the string is empty, either before or after resolving variables, we skip the encoding.
........... Usefulness: ...........
An example of why this feature may be useful is redirecting HTTP to HTTPS with something like:
"action": { "return": 301, "location": "https://${host}${uri}" }
..... Bugs: .....
This feature conflicts with the relevant RFCs in the following:
'$' is used for Unit variables, but '$' is a reserved character in a URI, to be used as a sub-delimiter. However, it's almost never used as that, and in fact, other parts of Unit already conflict with '$' being a reserved character for use as a sub-delimiter, so this is at least consistent in that sense. VBart suggested an easy workaround if we ever need it: adding a variable '$sign' which resolves to a literal '$'.
...... Notes: ......
An empty string is handled as if "location" wasn't specified at all, so no Location header is sent.
This is incorrect, and the code is slightly misleading.
The Location header consists of a URI-reference[1], which might be a relative one, which itself might consist of an empty string[2].
[1]: <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2> [2]: <https://stackoverflow.com/a/43338457>
Now that we have variables, it's more likely that an empty Location header will be requested, and we should handle it correctly.
I think in a future commit we should modify the code to allow differentiating between an unset "location" and an empty one, which should be treated as any other "location" string.
................. Testing (manual): .................
{ "listeners": { "*:80": { "pass": "routes/str" }, "*:81": { "pass": "routes/empty" }, "*:82": { "pass": "routes/var" }, "*:83": { "pass": "routes/enc-str" }, "*:84": { "pass": "routes/enc-var" } }, "routes": { "str": [ { "action": { "return": 301, "location": "foo" } } ], "empty": [ { "action": { "return": 301, "location": "" } } ], "var": [ { "action": { "return": 301, "location": "$host" } } ], "enc-str": [ { "action": { "return": 301, "location": "f%23o#o" } } ], "enc-var": [ { "action": { "return": 301, "location": "f%23o${host}#o" } } ] } }
$ curl --dump-header - localhost:80 HTTP/1.1 301 Moved Permanently Location: foo Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:06 GMT Content-Length: 0
$ curl --dump-header - localhost:81 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:08 GMT Content-Length: 0
$ curl --dump-header - localhost:82 HTTP/1.1 301 Moved Permanently Location: localhost Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:15 GMT Content-Length: 0
$ curl --dump-header - -H "Host: bar" localhost:82 HTTP/1.1 301 Moved Permanently Location: bar Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:82 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:29 GMT Content-Length: 0
$ curl --dump-header - localhost:83 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:44 GMT Content-Length: 0
$ curl --dump-header - -H "Host: alx" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23oalx#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:52 GMT Content-Length: 0
$ curl --dump-header - -H "Host: a#l%23x" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%2523oa#l%2523x%23o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:23:09 GMT Content-Length: 0
$ curl --dump-header - -H "Host: b##ar" localhost:82 HTTP/1.1 301 Moved Permanently Location: b#%23ar Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:25:01 GMT Content-Length: 0 diff 2081:c68e6afffb84 Tue Apr 05 09:47:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Supporting variables in "location".
............ Description: ............
Before this commit, the encoded URI could be calculated at configuration time. Now, since variables can only be resolved at request time, we have different situations:
- "location" contains no variables:
In this case, we still encode the URI in the conf structure, at configuration time, and then we just copy the resulting string to the ctx structure at request time.
- "location" contains variables:
In this case, we compile the var string at configure time, then when we resolve it at request time, and then we encode the string.
In both cases, as was being done before, if the string is empty, either before or after resolving variables, we skip the encoding.
........... Usefulness: ...........
An example of why this feature may be useful is redirecting HTTP to HTTPS with something like:
"action": { "return": 301, "location": "https://${host}${uri}" }
..... Bugs: .....
This feature conflicts with the relevant RFCs in the following:
'$' is used for Unit variables, but '$' is a reserved character in a URI, to be used as a sub-delimiter. However, it's almost never used as that, and in fact, other parts of Unit already conflict with '$' being a reserved character for use as a sub-delimiter, so this is at least consistent in that sense. VBart suggested an easy workaround if we ever need it: adding a variable '$sign' which resolves to a literal '$'.
...... Notes: ......
An empty string is handled as if "location" wasn't specified at all, so no Location header is sent.
This is incorrect, and the code is slightly misleading.
The Location header consists of a URI-reference[1], which might be a relative one, which itself might consist of an empty string[2].
[1]: <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2> [2]: <https://stackoverflow.com/a/43338457>
Now that we have variables, it's more likely that an empty Location header will be requested, and we should handle it correctly.
I think in a future commit we should modify the code to allow differentiating between an unset "location" and an empty one, which should be treated as any other "location" string.
................. Testing (manual): .................
{ "listeners": { "*:80": { "pass": "routes/str" }, "*:81": { "pass": "routes/empty" }, "*:82": { "pass": "routes/var" }, "*:83": { "pass": "routes/enc-str" }, "*:84": { "pass": "routes/enc-var" } }, "routes": { "str": [ { "action": { "return": 301, "location": "foo" } } ], "empty": [ { "action": { "return": 301, "location": "" } } ], "var": [ { "action": { "return": 301, "location": "$host" } } ], "enc-str": [ { "action": { "return": 301, "location": "f%23o#o" } } ], "enc-var": [ { "action": { "return": 301, "location": "f%23o${host}#o" } } ] } }
$ curl --dump-header - localhost:80 HTTP/1.1 301 Moved Permanently Location: foo Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:06 GMT Content-Length: 0
$ curl --dump-header - localhost:81 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:08 GMT Content-Length: 0
$ curl --dump-header - localhost:82 HTTP/1.1 301 Moved Permanently Location: localhost Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:15 GMT Content-Length: 0
$ curl --dump-header - -H "Host: bar" localhost:82 HTTP/1.1 301 Moved Permanently Location: bar Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:82 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:29 GMT Content-Length: 0
$ curl --dump-header - localhost:83 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:44 GMT Content-Length: 0
$ curl --dump-header - -H "Host: alx" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23oalx#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:52 GMT Content-Length: 0
$ curl --dump-header - -H "Host: a#l%23x" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%2523oa#l%2523x%23o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:23:09 GMT Content-Length: 0
$ curl --dump-header - -H "Host: b##ar" localhost:82 HTTP/1.1 301 Moved Permanently Location: b#%23ar Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:25:01 GMT Content-Length: 0 diff 2081:c68e6afffb84 Tue Apr 05 09:47:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Supporting variables in "location".
............ Description: ............
Before this commit, the encoded URI could be calculated at configuration time. Now, since variables can only be resolved at request time, we have different situations:
- "location" contains no variables:
In this case, we still encode the URI in the conf structure, at configuration time, and then we just copy the resulting string to the ctx structure at request time.
- "location" contains variables:
In this case, we compile the var string at configure time, then when we resolve it at request time, and then we encode the string.
In both cases, as was being done before, if the string is empty, either before or after resolving variables, we skip the encoding.
........... Usefulness: ...........
An example of why this feature may be useful is redirecting HTTP to HTTPS with something like:
"action": { "return": 301, "location": "https://${host}${uri}" }
..... Bugs: .....
This feature conflicts with the relevant RFCs in the following:
'$' is used for Unit variables, but '$' is a reserved character in a URI, to be used as a sub-delimiter. However, it's almost never used as that, and in fact, other parts of Unit already conflict with '$' being a reserved character for use as a sub-delimiter, so this is at least consistent in that sense. VBart suggested an easy workaround if we ever need it: adding a variable '$sign' which resolves to a literal '$'.
...... Notes: ......
An empty string is handled as if "location" wasn't specified at all, so no Location header is sent.
This is incorrect, and the code is slightly misleading.
The Location header consists of a URI-reference[1], which might be a relative one, which itself might consist of an empty string[2].
[1]: <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2> [2]: <https://stackoverflow.com/a/43338457>
Now that we have variables, it's more likely that an empty Location header will be requested, and we should handle it correctly.
I think in a future commit we should modify the code to allow differentiating between an unset "location" and an empty one, which should be treated as any other "location" string.
................. Testing (manual): .................
{ "listeners": { "*:80": { "pass": "routes/str" }, "*:81": { "pass": "routes/empty" }, "*:82": { "pass": "routes/var" }, "*:83": { "pass": "routes/enc-str" }, "*:84": { "pass": "routes/enc-var" } }, "routes": { "str": [ { "action": { "return": 301, "location": "foo" } } ], "empty": [ { "action": { "return": 301, "location": "" } } ], "var": [ { "action": { "return": 301, "location": "$host" } } ], "enc-str": [ { "action": { "return": 301, "location": "f%23o#o" } } ], "enc-var": [ { "action": { "return": 301, "location": "f%23o${host}#o" } } ] } }
$ curl --dump-header - localhost:80 HTTP/1.1 301 Moved Permanently Location: foo Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:06 GMT Content-Length: 0
$ curl --dump-header - localhost:81 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:08 GMT Content-Length: 0
$ curl --dump-header - localhost:82 HTTP/1.1 301 Moved Permanently Location: localhost Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:15 GMT Content-Length: 0
$ curl --dump-header - -H "Host: bar" localhost:82 HTTP/1.1 301 Moved Permanently Location: bar Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:82 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:29 GMT Content-Length: 0
$ curl --dump-header - localhost:83 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:44 GMT Content-Length: 0
$ curl --dump-header - -H "Host: alx" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23oalx#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:52 GMT Content-Length: 0
$ curl --dump-header - -H "Host: a#l%23x" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%2523oa#l%2523x%23o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:23:09 GMT Content-Length: 0
$ curl --dump-header - -H "Host: b##ar" localhost:82 HTTP/1.1 301 Moved Permanently Location: b#%23ar Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:25:01 GMT Content-Length: 0 diff 2081:c68e6afffb84 Tue Apr 05 09:47:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Supporting variables in "location".
............ Description: ............
Before this commit, the encoded URI could be calculated at configuration time. Now, since variables can only be resolved at request time, we have different situations:
- "location" contains no variables:
In this case, we still encode the URI in the conf structure, at configuration time, and then we just copy the resulting string to the ctx structure at request time.
- "location" contains variables:
In this case, we compile the var string at configure time, then when we resolve it at request time, and then we encode the string.
In both cases, as was being done before, if the string is empty, either before or after resolving variables, we skip the encoding.
........... Usefulness: ...........
An example of why this feature may be useful is redirecting HTTP to HTTPS with something like:
"action": { "return": 301, "location": "https://${host}${uri}" }
..... Bugs: .....
This feature conflicts with the relevant RFCs in the following:
'$' is used for Unit variables, but '$' is a reserved character in a URI, to be used as a sub-delimiter. However, it's almost never used as that, and in fact, other parts of Unit already conflict with '$' being a reserved character for use as a sub-delimiter, so this is at least consistent in that sense. VBart suggested an easy workaround if we ever need it: adding a variable '$sign' which resolves to a literal '$'.
...... Notes: ......
An empty string is handled as if "location" wasn't specified at all, so no Location header is sent.
This is incorrect, and the code is slightly misleading.
The Location header consists of a URI-reference[1], which might be a relative one, which itself might consist of an empty string[2].
[1]: <https://www.rfc-editor.org/rfc/rfc7231#section-7.1.2> [2]: <https://stackoverflow.com/a/43338457>
Now that we have variables, it's more likely that an empty Location header will be requested, and we should handle it correctly.
I think in a future commit we should modify the code to allow differentiating between an unset "location" and an empty one, which should be treated as any other "location" string.
................. Testing (manual): .................
{ "listeners": { "*:80": { "pass": "routes/str" }, "*:81": { "pass": "routes/empty" }, "*:82": { "pass": "routes/var" }, "*:83": { "pass": "routes/enc-str" }, "*:84": { "pass": "routes/enc-var" } }, "routes": { "str": [ { "action": { "return": 301, "location": "foo" } } ], "empty": [ { "action": { "return": 301, "location": "" } } ], "var": [ { "action": { "return": 301, "location": "$host" } } ], "enc-str": [ { "action": { "return": 301, "location": "f%23o#o" } } ], "enc-var": [ { "action": { "return": 301, "location": "f%23o${host}#o" } } ] } }
$ curl --dump-header - localhost:80 HTTP/1.1 301 Moved Permanently Location: foo Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:06 GMT Content-Length: 0
$ curl --dump-header - localhost:81 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:08 GMT Content-Length: 0
$ curl --dump-header - localhost:82 HTTP/1.1 301 Moved Permanently Location: localhost Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:15 GMT Content-Length: 0
$ curl --dump-header - -H "Host: bar" localhost:82 HTTP/1.1 301 Moved Permanently Location: bar Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:82 HTTP/1.1 301 Moved Permanently Server: Unit/1.27.0 Date: Thu, 07 Apr 2022 23:30:29 GMT Content-Length: 0
$ curl --dump-header - localhost:83 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:23 GMT Content-Length: 0
$ curl --dump-header - -H "Host: " localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23o#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:44 GMT Content-Length: 0
$ curl --dump-header - -H "Host: alx" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%23oalx#o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:22:52 GMT Content-Length: 0
$ curl --dump-header - -H "Host: a#l%23x" localhost:84 HTTP/1.1 301 Moved Permanently Location: f%2523oa#l%2523x%23o Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:23:09 GMT Content-Length: 0
$ curl --dump-header - -H "Host: b##ar" localhost:82 HTTP/1.1 301 Moved Permanently Location: b#%23ar Server: Unit/1.27.0 Date: Sat, 09 Apr 2022 11:25:01 GMT Content-Length: 0
|
H A D | nxt_http_static.c | diff 2619:24bba891e97e Wed Jan 17 17:18:00 UTC 2024 Andrew Clayton <a.clayton@nginx.com> HTTP: Remove short read check in nxt_http_static_buf_completion()
On GH, @tonychuuy reported an issue when using Units 'share' action they would get the following error in the unit log
2024/01/15 17:53:41 [error] 49#52 *103 file "/var/www/html/public/vendor/telescope/app.css" has changed while sending response to a client
This would happen when trying to serve files over a certain size and the requested file would not be sent.
This is due to a somewhat bogus check in nxt_http_static_buf_completion()
I say bogus because it's not clear what the check is trying to accomplish and the error message is not entirely accurate either.
The check in question goes like
n = pread(file->fd, buf, size, offset); return n; ... if (n != size) { if (n >= 0) { /* log file changed error and finish */
/* >> Problem is here << */ }
/* log general error and finish */ }
If the number of bytes read is not what we asked for and is > -1 (i.e not an error) then it says the file has changed, but really it only checks if the file has _shrunk_ (we can't get back _more_ bytes than we asked for) since it was stat'd.
This is what happens
recvfrom(22, "GET /tfile HTTP/1.1\r\nHost: local"..., 2048, 0, NULL, NULL) = 82 openat(AT_FDCWD, "/mnt/9p/tfile", O_RDONLY|O_NONBLOCK) = 23 newfstatat(23, "", {st_mode=S_IFREG|0644, st_size=149922, ...}, AT_EMPTY_PATH) = 0
We get a request from a client, open the requested file and stat(2) it to get the file size.
We would then go into a pread/writev loop reading the file data and sending it to the client until it's all been sent.
However what was happening in this case was this (showing a dummy file of 149922 bytes)
pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 131072, 0) = 61440 write(2, "2024/01/17 15:30:50 [error] 1849"..., 109) = 109
We wanted to read 131072 bytes but only read 61440 bytes, the above check triggered and the file transfer was aborted and the above error message logged.
Normally for a regular file you will only get less bytes than asked for if the read call is interrupted by a signal or you're near the end of file.
There is however at least another situation where this may happen, if the file in question is being served from a network filesystem.
It turns out that was indeed the case here, the files where being served over the 9P filesystem protocol. Unit was running in a docker container in an Ubuntu VM under Windows/WSL2 and the files where being passed through to the VM from Windows over 9P.
Whatever the intention of this check, it is clearly causing issues in real world scenarios.
If it was really desired to check if the had changed since it was opened/stat'd then it would require a different methodology and be a patch for another day. But as it stands this current check does more harm than good, so lets just remove it.
With it removed we now get for the above test file
recvfrom(22, "GET /tfile HTTP/1.1\r\nHost: local"..., 2048, 0, NULL, NULL) = 82 openat(AT_FDCWD, "/mnt/9p/tfile", O_RDONLY|O_NONBLOCK) = 23 newfstatat(23, "", {st_mode=S_IFREG|0644, st_size=149922, ...}, AT_EMPTY_PATH) = 0 mmap(NULL, 135168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f367817b000 pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 131072, 0) = 61440 pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 18850, 61440) = 18850 writev(22, [{iov_base="HTTP/1.1 200 OK\r\nLast-Modified: "..., iov_len=171}, {iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=61440}, {iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=18850}], 3) = 80461 pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 69632, 80290) = 61440 pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 8192, 141730) = 8192 close(23) = 0 writev(22, [{iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=61440}, {iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=8192}], 2) = 69632
So we can see we do two pread(2)s's and a writev(2), then another two pread(2)s and another writev(2) and all the file data has been read and sent to the client.
Reported-by: tonychuuy <https://github.com/tonychuuy> Link: <https://en.wikipedia.org/wiki/9P_(protocol)> Fixes: 08a8d1510 ("Basic support for serving static files.") Closes: https://github.com/nginx/unit/issues/1064 Reviewed-by: Zhidao Hong <z.hong@f5.com> Reviewed-by: Andrei Zeliankou <zelenkov@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com> diff 2619:24bba891e97e Wed Jan 17 17:18:00 UTC 2024 Andrew Clayton <a.clayton@nginx.com> HTTP: Remove short read check in nxt_http_static_buf_completion()
On GH, @tonychuuy reported an issue when using Units 'share' action they would get the following error in the unit log
2024/01/15 17:53:41 [error] 49#52 *103 file "/var/www/html/public/vendor/telescope/app.css" has changed while sending response to a client
This would happen when trying to serve files over a certain size and the requested file would not be sent.
This is due to a somewhat bogus check in nxt_http_static_buf_completion()
I say bogus because it's not clear what the check is trying to accomplish and the error message is not entirely accurate either.
The check in question goes like
n = pread(file->fd, buf, size, offset); return n; ... if (n != size) { if (n >= 0) { /* log file changed error and finish */
/* >> Problem is here << */ }
/* log general error and finish */ }
If the number of bytes read is not what we asked for and is > -1 (i.e not an error) then it says the file has changed, but really it only checks if the file has _shrunk_ (we can't get back _more_ bytes than we asked for) since it was stat'd.
This is what happens
recvfrom(22, "GET /tfile HTTP/1.1\r\nHost: local"..., 2048, 0, NULL, NULL) = 82 openat(AT_FDCWD, "/mnt/9p/tfile", O_RDONLY|O_NONBLOCK) = 23 newfstatat(23, "", {st_mode=S_IFREG|0644, st_size=149922, ...}, AT_EMPTY_PATH) = 0
We get a request from a client, open the requested file and stat(2) it to get the file size.
We would then go into a pread/writev loop reading the file data and sending it to the client until it's all been sent.
However what was happening in this case was this (showing a dummy file of 149922 bytes)
pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 131072, 0) = 61440 write(2, "2024/01/17 15:30:50 [error] 1849"..., 109) = 109
We wanted to read 131072 bytes but only read 61440 bytes, the above check triggered and the file transfer was aborted and the above error message logged.
Normally for a regular file you will only get less bytes than asked for if the read call is interrupted by a signal or you're near the end of file.
There is however at least another situation where this may happen, if the file in question is being served from a network filesystem.
It turns out that was indeed the case here, the files where being served over the 9P filesystem protocol. Unit was running in a docker container in an Ubuntu VM under Windows/WSL2 and the files where being passed through to the VM from Windows over 9P.
Whatever the intention of this check, it is clearly causing issues in real world scenarios.
If it was really desired to check if the had changed since it was opened/stat'd then it would require a different methodology and be a patch for another day. But as it stands this current check does more harm than good, so lets just remove it.
With it removed we now get for the above test file
recvfrom(22, "GET /tfile HTTP/1.1\r\nHost: local"..., 2048, 0, NULL, NULL) = 82 openat(AT_FDCWD, "/mnt/9p/tfile", O_RDONLY|O_NONBLOCK) = 23 newfstatat(23, "", {st_mode=S_IFREG|0644, st_size=149922, ...}, AT_EMPTY_PATH) = 0 mmap(NULL, 135168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f367817b000 pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 131072, 0) = 61440 pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 18850, 61440) = 18850 writev(22, [{iov_base="HTTP/1.1 200 OK\r\nLast-Modified: "..., iov_len=171}, {iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=61440}, {iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=18850}], 3) = 80461 pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 69632, 80290) = 61440 pread64(23, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 8192, 141730) = 8192 close(23) = 0 writev(22, [{iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=61440}, {iov_base="\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., iov_len=8192}], 2) = 69632
So we can see we do two pread(2)s's and a writev(2), then another two pread(2)s and another writev(2) and all the file data has been read and sent to the client.
Reported-by: tonychuuy <https://github.com/tonychuuy> Link: <https://en.wikipedia.org/wiki/9P_(protocol)> Fixes: 08a8d1510 ("Basic support for serving static files.") Closes: https://github.com/nginx/unit/issues/1064 Reviewed-by: Zhidao Hong <z.hong@f5.com> Reviewed-by: Andrei Zeliankou <zelenkov@nginx.com> Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
|
H A D | nxt_http_variables.c | diff 2110:048281cd3d73 Thu May 26 12:38:00 UTC 2022 Alejandro Colomar <alx.manpages@gmail.com> Var: Added $request_uri (as in NGINX).
This supports a new variable $request_uri that contains the path and the query (See RFC 3986, section 3). Its contents are percent encoded. This is useful for example to redirect HTTP to HTTPS:
{ "return": "301", "location": "https://$host$request_uri" }
When <http://example.com/foo%23bar?baz> is requested, the server redirects to <https://example.com/foo%23bar?baz>.
===
Testing:
//diff --git a/src/nxt_http_return.c b/src/nxt_http_return.c //index 82c9156..adeb3a1 100644 //--- a/src/nxt_http_return.c //+++ b/src/nxt_http_return.c //@@ -196,6 +196,7 @@ nxt_http_return_send_ready(nxt_task_t *task, void *obj, void *data) // field->value = ctx->encoded.start; // field->value_length = ctx->encoded.length; // } //+ fprintf(stderr, "ALX: target[%1$i]: <%2$.*1$s>\n", (int)r->target.length, r->target.start); // // r->state = &nxt_http_return_send_state; //
{ "listeners": { "*:81": { "pass": "routes/ru" } },
"routes": { "ru": [{ "action": { "return": 301, "location": "$request_uri" } }] } }
$ curl -i http://localhost:81/*foo%2Abar?baz#arg HTTP/1.1 301 Moved Permanently Location: /*foo%2Abar?baz Server: Unit/1.27.0 Date: Mon, 30 May 2022 16:04:30 GMT Content-Length: 0
$ sudo cat /usr/local/unit.log | grep ALX ALX: target[15]: </*foo%2Abar?baz>
|
H A D | nxt_main_process.c | diff 342:82c2825a617a Wed Oct 04 11:58:00 UTC 2017 Max Romanov <max.romanov@nginx.com> Using engine memiory pool for port write allocations.
To allow use port from different threads, the first step is to avoid using port's memory pool for temporary allocations required to send data through the port. Including but not limited by: - buffers for data; - send message structures; - new mmap fd notifications;
It is still safe to use port memory pool for incoming buffers allocations because recieve operation bound to single thread.
|
H A D | nxt_port.c | diff 342:82c2825a617a Wed Oct 04 11:58:00 UTC 2017 Max Romanov <max.romanov@nginx.com> Using engine memiory pool for port write allocations.
To allow use port from different threads, the first step is to avoid using port's memory pool for temporary allocations required to send data through the port. Including but not limited by: - buffers for data; - send message structures; - new mmap fd notifications;
It is still safe to use port memory pool for incoming buffers allocations because recieve operation bound to single thread.
|
H A D | nxt_port.h | diff 82:4be7aabeb2d1 Fri Jun 23 16:20:00 UTC 2017 Max Romanov <max.romanov@nginx.com> Moved message size to nxt_port_recv_msg_t for convenience.
|
H A D | nxt_port_memory.c | diff 342:82c2825a617a Wed Oct 04 11:58:00 UTC 2017 Max Romanov <max.romanov@nginx.com> Using engine memiory pool for port write allocations.
To allow use port from different threads, the first step is to avoid using port's memory pool for temporary allocations required to send data through the port. Including but not limited by: - buffers for data; - send message structures; - new mmap fd notifications;
It is still safe to use port memory pool for incoming buffers allocations because recieve operation bound to single thread. diff 82:4be7aabeb2d1 Fri Jun 23 16:20:00 UTC 2017 Max Romanov <max.romanov@nginx.com> Moved message size to nxt_port_recv_msg_t for convenience.
|
H A D | nxt_port_memory.h | diff 82:4be7aabeb2d1 Fri Jun 23 16:20:00 UTC 2017 Max Romanov <max.romanov@nginx.com> Moved message size to nxt_port_recv_msg_t for convenience.
|
H A D | nxt_port_socket.c | diff 342:82c2825a617a Wed Oct 04 11:58:00 UTC 2017 Max Romanov <max.romanov@nginx.com> Using engine memiory pool for port write allocations.
To allow use port from different threads, the first step is to avoid using port's memory pool for temporary allocations required to send data through the port. Including but not limited by: - buffers for data; - send message structures; - new mmap fd notifications;
It is still safe to use port memory pool for incoming buffers allocations because recieve operation bound to single thread. diff 82:4be7aabeb2d1 Fri Jun 23 16:20:00 UTC 2017 Max Romanov <max.romanov@nginx.com> Moved message size to nxt_port_recv_msg_t for convenience.
|
H A D | nxt_router.c | diff 342:82c2825a617a Wed Oct 04 11:58:00 UTC 2017 Max Romanov <max.romanov@nginx.com> Using engine memiory pool for port write allocations.
To allow use port from different threads, the first step is to avoid using port's memory pool for temporary allocations required to send data through the port. Including but not limited by: - buffers for data; - send message structures; - new mmap fd notifications;
It is still safe to use port memory pool for incoming buffers allocations because recieve operation bound to single thread.
|
H A D | nxt_runtime.c | diff 342:82c2825a617a Wed Oct 04 11:58:00 UTC 2017 Max Romanov <max.romanov@nginx.com> Using engine memiory pool for port write allocations.
To allow use port from different threads, the first step is to avoid using port's memory pool for temporary allocations required to send data through the port. Including but not limited by: - buffers for data; - send message structures; - new mmap fd notifications;
It is still safe to use port memory pool for incoming buffers allocations because recieve operation bound to single thread.
|
/unit/src/test/ |
H A D | nxt_http_parse_test.c | diff 574:1ec560778942 Thu Mar 15 17:52:00 UTC 2018 Valentin Bartenev <vbart@nginx.com> HTTP parser: fixed parsing of field values ending with space.
This closes #82 issue on GitHub. diff 454:82c827d1cd13 Tue Jan 09 13:50:00 UTC 2018 Valentin Bartenev <vbart@nginx.com> HTTP parser: allowing underscore in header field names.
|