Comments (30)
There appears to be a bug/typo, which I'll clarify. In short, the bug is in newSessionController, where we should set these to nil:
Lines 57 to 58 in df6e4c8
The SessionController
was designed to manage all session-related operations. Initially, it had ownership over extensions.
But this posed a challenge. The ClientHelloSpec might have a different set of PSK/Session extensions. For clarity, the original design had the SessionController
take charge of the extensions, overwriting those from the spec. This leads to the design that newSessionController
produces a non-nil session and a psk extension. A downside of this approach was that it altered the signature of a public method, and we didn't have a clear vision for v2.0 yet.
To circumvent this, I pivoted to a strategy where the Spec owns the extensions, ensuring the method signature remains unchanged. However, users should be able to override psk/session extensions before invoking ApplyPreset
. These overriding extensions should exclusively reside in the SessionController
. In such cases, the SessionController
have ownership over the extensions.
This necessitated the creation of the syncSessionExts
method to discern which extension takes precedence. The guiding principle is straightforward:
- If both the user and the spec offer an extension, prioritize the user's.
- If the spec lacks an extension and the user attempts to provide one, it's deemed an error.
- In the absence of a user-provided extension, revert to the spec's version, if available.
Yet, during this overhaul, I overlooked the newSessionController
method. As illustrated in the comments, deep in my mind I believe that the extension would default to nil unless overridden by the user:
Lines 271 to 273 in df6e4c8
from utls.
Interesting observation. I don't believe we are ready to parse raw byte ClientHello into PSK yet. On the other hand it is theoretically not possible to resume the connection with just the PSK ticket and binder from raw bytes.
That said, it might worth debugging and fixing to make at least FakePreSharedKeyExtension work with raw bytes. But note that in the end, unexpected things would happen, such as server rejecting you for submitting a valid ticket with an incorrect binder, etc.
Tagging @3andne for visibility.
from utls.
To clarify on the intended behaviors:
- If
RealPSKResumption
is set to true: parsing from raw bytes will discard the ticket and binders from the raw bytes, but use the available ones fromClientSessionCache
instead. - If
RealPSKResumption
is false: parsing from raw bytes with legitimate PSK tickets and binders IS GUARANTEED TO FAIL, given that:- the server will check the ticket and find it is legitimate, and proceed with verifying the binder. If we are sending binders from the original raw bytes (that's the intended behavior), the binder will not match the expected value which is calculated from all previous bytes in ClientHello.
- If we were to re-calculated binders, it will be a different challenge for us to find the shared secret for that specific ticket.
from utls.
Ok, i'll dive deeper into FakePreSharedKeyExtension
and see if i can fix what ever is broken with it (if a PSK is present) .. 🤞
from utls.
@VeNoMouS No problem! Thanks for your efforts!
from utls.
Sorry about the delay in getting a PR sorted... life and all that 😄
from utls.
Addtionally ...
Noticed something odd happening when using FakePreSharedKeyExtension
its calling the wrong extension to Read()
, due to this
from utls.
Given all the complexities, I would encourage avoiding reusing raw bytes with PSK, either way it will not work as you expect, either we need to use a different PSK ticket/binder, or the connection will fail outright.
from utls.
FWIW, it use to work, prior to the change (when genericExtension 41 was used etc)
from utls.
prior to the change (when genericExtension 41 was used etc)
Yep, and in that case, you would actually expect the server to return an error. Just based on my test result, when I use a legitimate PSK Identity with its original binder, the server gives me an error (due to failing the binder check).
Have you actually observed any other behaviors? That would be really intriguing.
And a quick patch here could be just roll back to use GenericExtension
by removing the Writer
implementation on PSK extensions. Currently, I do not really have a good idea for how to we handle this appropriately ... or, any suggestions?
from utls.
Yea i started doing that last night, tho ran into an issue with unable to decrypt error , trying to work out what is going on
from utls.
unable to decrypt error
I don't remember exactly what error did I receive when I was testing. But there are essentially two possible outcomes:
- Server rejects your ClientHello (you get a remote error) due to incorrect Binder (Random/GREASE/KeyShare has changed) and binder is not re-calculated
- Server didn't catch that issue and accepted your resumption. Then sent you an encrypted message. You don't REALLY have the selected PSK Identity (more specifically, the corresponding key) so could not decrypt it.
from utls.
- Server rejects your ClientHello (you get a remote error) due to incorrect Binder (Random/GREASE/KeyShare has changed) and binder is not re-calculated
This just gave me hint , when i was testing my custom spec i had grease place holder... so i'll test with static grease id's and let you know on that
Switching back to rawBytes and using a genericExtenstion 41 here worked, the PSK loaded from raw bytes and was present on the hello
Not sure why that's in a switch
either as its only checking one case ..
from utls.
@gaukas what do you think about doing something with this
and replace it with
if ext != nil && ok { // known extension and implements TLSExtensionWriter properly
switch extension {
case extensionPreSharedKey:
// PSK extension, need to see if we do real or fake PSK
if realPSK {
extWriter = &UtlsPreSharedKeyExtension{}
} else {
if len(extData) == 0 {
extWriter = &FakePreSharedKeyExtension{}
} else {
extWriter = &GenericExtension{Id: 41}
}
}
case extensionSupportedVersions:
chs.TLSVersMin = 0
chs.TLSVersMax = 0
}
if _, err := extWriter.Write(extData); err != nil {
return err
}
chs.Extensions = append(chs.Extensions, extWriter)
And adding a simple write to the GenericExtension in the tls_extensions
func (e *GenericExtension) Write(b []byte) (int, error) {
e.Data = make([]byte, len(b))
n := copy(e.Data, b)
return n, nil
}
So if the realPSK
is false
(default) it uses FakePreSharedKeyExtension
and if the PSK is populated in the extension data it uses the genericExtension 41, ReadTLSExtensions
is only called from FromRaw
from utls.
FakePreSharedKeyExtension
is actually just a GenericExtension
but with PSK's data fields (identity and binders) explicitly defined. So entirely removing the existence of FakePreSharedKeyExtension
or fixing the behavior when FakePreSharedKeyExtension
would suffice. There's no reason to keep them both.
from utls.
Just an update on this, (sorry been busy), I can see something is happening when FakePreSharedKeyExtension
does a Read
, I can confirm Write
does create the Identities
and Binders
, but when Read
is called they are gone for some reason, so I'm trying to work out what is causing that to occur...
Sorry it appears before that, it Does a Write
then a Len
, and its gone by the time it hits Len
FakePreSharedKeyExtension Write
Dumping e.Identities: [{[133 74 171 187 44 97 94 24 1 4 49 205 230 205 54 184 229 24 74 171 116 211 248 230 99 190 254 164 80 214 134 51 173 79 189 248 209 74 30 39 144 224 74 52 52 128 110 234 152 231 163 63 229 194 197 231 76 68 219 167 124 123 29 163 228 117 185 253 99 69 235 90 213 87 0 47 149 243 111 119 26 64 115 109 227 121 180 207 231 65 211 81 212 18 25 99 243 90 187 212 180 193 162 193 194 94 35 74 179 12 178 183 134] 2239559142}]
Dumping e.Binders: [[255 152 173 142 128 210 182 214 210 243 227 171 115 165 68 122 254 155 121 203 158 20 58 170 198 19 182 113 177 145 13 88]]
FakePreSharedKeyExtension Len()
Dumping e.Identities: []
Dumping e.Binders: []
Update:
Found the "problem" with the disappearing PSK... its in syncSessionExts
on this line
if we remove that line PSK is populated... I need to read the code to understand the impact of removing it
from utls.
Thanks for your detailed report and effort in investigation @VeNoMouS!
@3andne I would greatly appreciate if you would love to share some of your insights on this issue.
from utls.
What if we did something like the following for that line
if s.pskExtension.Len() != 0 || s.uconnRef.Extensions[i].Len() == 0 {
s.uconnRef.Extensions[i] = s.pskExtension
}
from utls.
It feels like that most likely you do not want to explicitly provide a PSK extension in case you are reading from raw bytes. So I would expect it to hit Line 273 instead.
Lines 271 to 277 in df6e4c8
from utls.
How do you feel about adding
case *FakePreSharedKeyExtension:
if len(ext.Identities) != 0 && len(ext.Binders) != 0 {
uconn.sessionController.overridePskExt(ext)
}
to here?
Line 2486 in df6e4c8
Obviously that's slower as it goes and calls all the sub functions to pull the whole extension again.. but...
Line 80 in df6e4c8
from utls.
I'm not sure if that's the correct usage of overridePskExt()... I might need to looks into this a bit.
from utls.
It feels like that most likely you do not want to explicitly provide a PSK extension in case you are reading from raw bytes. So I would expect it to hit Line 273 instead.
Lines 271 to 277 in df6e4c8
other option is simply...
if s.pskExtension == nil || ext.Len() != 0 {
Line 271 in df6e4c8
THe problem is s.pskExtension
isn't nil
, but then you get into a problem, of if the PSK is populated from raw bytes, and you want to pass your own extension, how do you overwrite that in the setup etc..
hence why i looked at overridePskExt
from utls.
Why would s.pskExtension be set in this case? (Or, why do we need the or) Where is it set and what is it set to?
from utls.
Why would s.pskExtension be set in this case? (Or, why do we need the or) Where is it set and what is it set to?
syncSessionExts
gets called from ApplyPreset
and when it creates the sessionController
, newSessionController
is called which creates an empty pskExtension
Lines 54 to 64 in df6e4c8
That said, could just make pskExtension
in newSessionController
nil
😄 (i tested that and it seems to work)
from utls.
i tested that and it seems to work
Have you also tested the psk examples to make sure they all still work?
Let's check this with 3andne first, to make sure we don't miss any part.
from utls.
Yea lets wait for @3andne because I can envision some issues with setting it to nil
... (or not)... that said overridePskExt
might be the better route...
lets wait and see
from utls.
IMHO
sessionTicketExt
and pskExtension
should both be nil
innewSessionController
, as there is heaps of sanity checks looking for nil
that have safe guards in place for it
from utls.
Hey @VeNoMouS , can you provide a minimal code snippet that can reproduce the behavior. I need your help on the context. Thanks!
from utls.
just use the raw bytes in the first comment
and use FingerprintClientHello()
with said raw bytes to make spec and a GET to https://tls.peet.ws/api/all
and make sure pre_shared_key (41)
is present in the extensions listed in the response
from utls.
Thanks for looking at this lengthy debug ticket @3andne 😄 , are you ok if I do a PR for newSessionController
and have
Lines 57 to 58 in df6e4c8
Defaulting to nil
? or do you want to do that?
from utls.
Related Issues (20)
- How to implement the "extensionEncryptedClientHello (boringssl) (65037)," extended HOT 23
- remote error: tls: missing extension HOT 5
- random Can not connect: tls: first record does not look like a TLS handshake error HOT 2
- problem with gorilla websocket package HOT 3
- Consider adding MarshalJson support for ClientHelloSpec HOT 5
- [BUG] ? StatusRequestV2Extension HOT 7
- Please, bump the major version number when you break the API HOT 5
- PSK resumption and ClientHelloRetry HOT 1
- Unable to set `OmitEmptyPsk` in `PreSharedKeyExtension` HOT 3
- Conn.readRecord(...) with multiple goroutine error HOT 1
- Cannot handshake with speed.hetzner.de HOT 4
- Cannot install in Docker base image alpine (package crypto/ecdh is not in GOROOT) HOT 5
- panic: tls: setSessionTicketExt failed: invalid state HOT 3
- Support for padding extension HOT 6
- feat: GREASE ECH Extension HOT 4
- bump Auto parrot for Firefox and Chrome
- bug: configuration for GREASE ECH parrot for Chrome 120 doesn't match BoringSSL HOT 7
- HelloFirefox* gets an ECDSA verification failure HOT 4
- FingerprintClientHello support for GREASE ECH extension
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from utls.