-
Notifications
You must be signed in to change notification settings - Fork 1
Memory Usage Tests
Testing against some known web sites. The main idea is to figure out howw much memory we need to run. In each case i was issuing requests for /robots.txt as it's the only file anyone should get with non-browser tool.
| hostname | 24KB | 12KB | 6KB | 4KB | 3KB | comments |
|---|---|---|---|---|---|---|
| localhost Nginx | ok | ok | ok | ok | ok | Self-signed cert. is small enough ot fit. |
| example.com | ok | ok | ok | ok | mem | Returns the same page on every URI. |
| api.github.com | ok | ok | ok | ok | mem | Only 180 bytes short to fit 2nd certificate into 3K. 3rd certificate was small enough. |
| google.com | ok | ok | ok | ok | mem | Google ignored record size limit and always sent ~1400 bytes with each fragment. |
| api.digitalocean.com | ok | ok | ok | ok | mem |
mem = ZTLS_ERR_OVERFLOW error
Certificate storage is a bottleneck. While certificates are being processed one-by-one, they still need to be kept in memory, while their individual parts are being verified and extracted. Extracted info is about 660 bytes, certificates themselves are about 1.4 ~ 1.8KB. What kills it is the fact that there is a chunk of the next certificate already decrypted from the last fragment, which in the worst case can take up to 512 bytes. I don't want to minimize the fragment size even further, so what options do we have:
- Use separate memory block for handshake data and stash certificate info there. This might result in 2KB being needed for application protocol, but 2KB + 2KB for handshake. In some use cases this migth be a preferred solution.
- Don't allocate array of 8 certificate slots, instead allocate them individually. This migth be easy enough.
- Parse certificates in smaller parts: [tbsCertificate.version ... tbsCertificate.subject], [tbsCertificate.subjectPublicKeyInfo], [tbsCertificate.extensions], [signatureAlgorithm ... signatureValue]. There are two problems: a) in order to compute hash you need to know the signatureAlgorithm, which is not going to be available until we receive the last bit of certificate, and b) we'll have to load certificates by 1 or 2 bytes as ASN.1 is not easy to parse with async socket API. ... also, i don't to do that.
So what exactly i'm going to do? No clue, but it's probably going to be a combination of #1 and #2.