unable to query TXT records when using a QUIC upstream - SERVFAIL

[earache]

When using dnsproxy with a quic upstream I'm unable to lookup TXT records.

quic upstream:

;  <<>> DiG 9.20.8 <<>> txt google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 57950
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;google.com.                    IN      TXT

;; Query time: 10 msec
;; SERVER: 10.0.0.1#53(10.0.0.1) (UDP)
;; WHEN: Sat Apr 19 10:46:46 ACST 2025
;; MSG SIZE  rcvd: 39

tls upstream:

; <<>> DiG 9.20.8 <<>> txt google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57195
;; flags: qr rd ra; QUERY: 1, ANSWER: 12, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;google.com.                    IN      TXT

;; ANSWER SECTION:
google.com.             3243    IN      TXT     "onetrust-domain-verification=de01ed21f2fa4d8781cbc3ffb89cf4ef"
google.com.             3243    IN      TXT     "v=spf1 include:_spf.google.com ~all"
google.com.             3243    IN      TXT     "docusign=1b0a6754-49b1-4db5-8540-d2c12664b289"
google.com.             3243    IN      TXT     "apple-domain-verification=30afIBcvSuDV2PLX"
google.com.             3243    IN      TXT     "facebook-domain-verification=22rm551cu4k0ab0bxsw536tlds4h95"
google.com.             3243    IN      TXT     "cisco-ci-domain-verification=479146de172eb01ddee38b1a455ab9e8bb51542ddd7f1fa298557dfa7b22d963"
google.com.             3243    IN      TXT     "google-site-verification=TV9-DBe4R80X4v0M4U_bd_J9cpOJM0nikft0jAgjmsQ"
google.com.             3243    IN      TXT     "google-site-verification=wD8N7i1JTNTkezJ49swvWW48f8_9xveREV4oB-0Hf5o"
google.com.             3243    IN      TXT     "docusign=05958488-4752-4ef2-95eb-aa7ba8a3bd0e"
google.com.             3243    IN      TXT     "globalsign-smime-dv=CDYX+XFHUw2wml6/Gb8+59BsH31KzUr6c1l2BPvqKX8="
google.com.             3243    IN      TXT     "MS=E4A68B9AB2BB9670BCE15412F62916164C0B20BB"
google.com.             3243    IN      TXT     "google-site-verification=4ibFUgB-wXLQ_S7vsXVomSTVamuOXBiVAzpR5IZ87D0"

;; Query time: 130 msec
;; SERVER: 10.0.0.1#53(10.0.0.1) (UDP)
;; WHEN: Sat Apr 19 10:46:03 ACST 2025
;; MSG SIZE  rcvd: 886

quic query direct (to adguard dns)

;; QUIC session (QUICv1)-(TLS1.3)-(ECDHE-X25519)-(ECDSA-SECP256R1-SHA256)-(AES-128-GCM)
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 0
;; Flags: qr rd ra; QUERY: 1; ANSWER: 12; AUTHORITY: 0; ADDITIONAL: 1

;; EDNS PSEUDOSECTION:
;; Version: 0; flags: ; UDP size: 0 B; ext-rcode: NOERROR
;; PADDING: 20 B

;; QUESTION SECTION:
;; google.com.                  IN      TXT

;; ANSWER SECTION:
google.com.             2385    IN      TXT     "apple-domain-verification=30afIBcvSuDV2PLX"
google.com.             2385    IN      TXT     "MS=E4A68B9AB2BB9670BCE15412F62916164C0B20BB"
google.com.             2385    IN      TXT     "google-site-verification=wD8N7i1JTNTkezJ49swvWW48f8_9xveREV4oB-0Hf5o"
google.com.             2385    IN      TXT     "facebook-domain-verification=22rm551cu4k0ab0bxsw536tlds4h95"
google.com.             2385    IN      TXT     "google-site-verification=TV9-DBe4R80X4v0M4U_bd_J9cpOJM0nikft0jAgjmsQ"
google.com.             2385    IN      TXT     "docusign=1b0a6754-49b1-4db5-8540-d2c12664b289"
google.com.             2385    IN      TXT     "cisco-ci-domain-verification=479146de172eb01ddee38b1a455ab9e8bb51542ddd7f1fa298557dfa7b22d963"
google.com.             2385    IN      TXT     "v=spf1 include:_spf.google.com ~all"
google.com.             2385    IN      TXT     "onetrust-domain-verification=de01ed21f2fa4d8781cbc3ffb89cf4ef"
google.com.             2385    IN      TXT     "docusign=05958488-4752-4ef2-95eb-aa7ba8a3bd0e"
google.com.             2385    IN      TXT     "globalsign-smime-dv=CDYX+XFHUw2wml6/Gb8+59BsH31KzUr6c1l2BPvqKX8="
google.com.             2385    IN      TXT     "google-site-verification=4ibFUgB-wXLQ_S7vsXVomSTVamuOXBiVAzpR5IZ87D0"

;; Received 910 B
;; Time 2025-04-19 10:55:29 ACST
;; From 94.140.14.49@853(QUIC) in 65.3 ms

[jhed9]

I can't reproduce the issue, it's working for me @earache.

I'm running the latest version from the master branch as of writing, f00be4d. My config is as follows, edns-addr changed for privacy.

max-go-routines: 0
ratelimit: 0
ratelimit-subnet-len-ipv4: 24
ratelimit-subnet-len-ipv6: 64
udp-buf-size: 8388608
upstream:
  - "quic://unfiltered.adguard-dns.com"
timeout: '10s'
edns: true
edns-addr: "2001:db8:45ba:e5c8:e975:2017:c773:27df"
verbose: false
cache: true
cache-size: 104857600

[jhed9]

Also, I'm curious, what app did you use the get the output in your third example? I'd like to use it. I like the crypto details in the header line.

[earache]

thx for confirming @jhed9.

the last query was with kdig which IIRC is part of knot-dns

This prompted me to do some more testing and noted that the queries work on a host e.g:
_dmarc.google.com

; <<>> DiG 9.20.8 <<>> txt _dmarc.google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 26155
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;_dmarc.google.com.             IN      TXT

;; ANSWER SECTION:
_dmarc.google.com.      300     IN      TXT     "v=DMARC1; p=reject; rua=mailto:mailauth-reports@google.com"

;; Query time: 299 msec
;; SERVER: 10.0.0.1#53(10.0.0.1) (UDP)
;; WHEN: Wed May 14 20:27:20 ACST 2025
;; MSG SIZE  rcvd: 117

but fail on the parent domain.
google.com

; <<>> DiG 9.20.8 <<>> txt google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 59711
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;google.com.                    IN      TXT

;; Query time: 30 msec
;; SERVER: 10.0.0.1#53(10.0.0.1) (UDP)
;; WHEN: Wed May 14 20:27:29 ACST 2025
;; MSG SIZE  rcvd: 39

[jhed9]

Thanks, I'm going to keep kdig in my toolkit.

I'm still not able to reproduce your issue on my end. If you run dnsproxy with the --verbose/-v parameter, does anything interesting show up in the logs?

I suspect your issue has something to do with the size of the response instead of it being a query to a parent domain. For example, the ietf.org base domain txt response is much smaller, does that response come through okay?

[earache]

Thanks again, no joy querying ietf.org either.
I created some short records on my own domains with similar results.

...and now I'm really confused:

freeman:~ $ dig txt google.com.au

; <<>> DiG 9.20.8 <<>> txt google.com.au
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64188
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;google.com.au.                 IN      TXT

;; ANSWER SECTION:
google.com.au.          300     IN      TXT     "v=spf1 -all"

;; Query time: 189 msec
;; SERVER: 10.0.0.1#53(10.0.0.1) (UDP)
;; WHEN: Thu May 15 21:16:28 ACST 2025
;; MSG SIZE  rcvd: 66

freeman:~ $ dig txt google.com

; <<>> DiG 9.20.8 <<>> txt google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 61693
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;google.com.                    IN      TXT

;; Query time: 39 msec
;; SERVER: 10.0.0.1#53(10.0.0.1) (UDP)
;; WHEN: Thu May 15 21:16:31 ACST 2025
;; MSG SIZE  rcvd: 39

[jhed9]

I think we're really going to need to see the logs when you run dnsproxy with the --verbose/-v parameter.

Your issue definitely seems to be related to the size of the TXT response. I thought maybe it was a lack of EDNS support somewhere in the network, which would limit the size the the response to 512 bytes. But txt ietf.org is a 304 byte response and you're not able to even receive that. You're able to receive a response of 66 bytes for txt google.com.au. So the problem is when the response size larger than a threshold in the range of 67 and 304 bytes.

If you want, you can mess around with different txt response sizes to see where it breaks down. dnscheck.tools offers a service that will send you an arbitrary amount of txt data, up to 4000 bytes. For example, you can query txt txtfill452.go.dnscheck.tools to get a 512 byte message response (comprised of 452 bytes of TXT data plus 60 bytes of header).

But I think we're really going to need to see the messages when you run dnsproxy with the --verbose/-v parameter. I'm thinking it's going to be a network problem, not a software problem, as I cannot reproduce the behavior on my end. But the messages from verbose mode dnsproxy may highlight otherwise.

[earache]

Further experimenting:
when my upstream is: quic://unfiltered.adguard-dns.com everything works fine
When using my adguard private dns quic://xxxxxxxx.d.adguard-dns.com the request fails
Querying directly from other app to my adguard private dns also fails
As far as I can tell from the logs, the query is successful and the upstream server returns an error.

dnsproxy is working as expected :)

[jhed9]

Nice sleuthing! After setting up an AdGuard private dns of my own, I got the exact same problems on my end. The TXT records at google.com and ietf.org failed to load, but google.com.au and _dmarc.google.com loaded normally. It was nice troubleshooting with you @earache, have a good day.

[ihurin]

@Chinaski1 can you check it out?