Error: headers_upstream: invalid entry 'x-kong-request-id' about kong

[ADD-SP]

@samugi @chobits Could you please take a look?

[chobits]

@samugi @chobits Could you please take a look?

Ok, I'll take a look

[chobits]

Hi @theMiddleBlue could you provide a detailed upgrade progress.

I could only reproduce a similar error message when downgrading from the version with request-id feature to release/3.4.x branch which does not support request id feature. Normally, upgrading will not trigger similar error messages.

My error message is Error: headers: invalid entry 'x-kong-request-id', but yours is headers_upstream: invalid entry 'x-kong-request-id', note that differences between headers and headers_upstream

---

The following is my investigation:

The reason why i found headers x-kong-request-id entry invalid is that there is x-kong-request-id in my .kong_env file while current version of kong gateway does not support request-id feature.

$ grep 'x-kong-request-id' -R /Users/xc/work/dev/kong/bazel-bin/build/kong-dev/kong/servroot/.kong_env
/Users/xc/work/dev/kong/bazel-bin/build/kong-dev/kong/servroot/.kong_env:headers = server_tokens,latency_tokens,x-kong-request-id

I think the issue I'm facing is because, previously, i started kong on a version that could generate x-kong-request-id in .kong_env file, then when i switched to release/3.4.x without the request_id feature, it will reported Error: headers: invalid entry 'x-kong-request-id' .

So my steps to reproduce it is as follows:

All the steps are runned under bazel environment

1. build and start kong in some branch with the request-id feature supported, then stop kong
    * make build-venv
    * start_services
    * kong migrations bootstrap
    * kong start
    * kong stop
3. checkout to `release/3.4.x`
    * git fetch origin release/3.4.x
    * git checkout release/3.4.x
5. build and start kong in `release/3.4.x` branch, then will find the error message
    

[chobits]

A simple way to resolve your problem is to remove x-kong-request-id from your .kong_env file, or just remove this file.

Do not remove this file before stopping kong, otherwise it will report error no this file:....kong_env, then yourkong stop will fail. If you encounter the error message of kong stop, you could manually stop nginx process of kong gateway.

[samugi]

A simple way to resolve your problem is to remove x-kong-request-id from your .kong_env file [...]

thank you @chobits for the investigation, I just want to highlight that the workaround described is only applicable to versions of Kong that are expected to not support request-id, but that is not the case on 3.5 where it is supported. We will need additional details on steps to reproduce, Kong deployment details and configuration used by @theMiddleBlue in order to determine the root cause. Thanks.

[github-actions]

This issue is marked as stale because it has been open for 14 days with no activity.

[github-actions]

Dear contributor,

We are automatically closing this issue because it has not seen any activity for three weeks.
We're sorry that your issue could not be resolved. If any new information comes up that could
help resolving it, please feel free to reopen it.

Your contribution is greatly appreciated!

Please have a look
our pledge to the community
for more information.

Sincerely,
Your Kong Gateway team