Can't resolve big domain lists

[oldDuDe124]

Hi
I'm trying to resolve a wordlist containing roughly 2.2 million domains , i know some domains must be resolved and are valid but when I pass the list to dnsx it just doesn't seem to be resolving them and finishes the scan after 10 seconds.
I've tried using stdin input like:
cat list.txt | dnsx -silent
and also using -l option:
dnsx -l list.txt -silent
but no luck!
I also tried reducing the list to 1k lines and it worked. so I guess the size of the list is the main problem here.

[MetzinAround]

Does anything happen when you use -stats during a run?

[ehsandeep]

@oldDuDe124 do you see any error that you can share when you run with -v option

[NotoriousRebel]

Same error occurs with tlsx as it does here, when giving it a file with IP ranges. At least 60,000+ IPs in total via a few subnets causes the process to get killed after around 10 seconds of running. Running with -v does not give any errors, but should be easy to replicate as you just need a few subnets in a file. For tlsx: tlsx -c 72 -v -l ip_ranges.txt -ro -san and dnsx: dnsx -t 72 -l ip_ranges.txt -ptr -resp-only May just be caused by too many IP ranges being expanded at once, I notice a huge spike in CPU usage before crashing.

[hastalamuerte]

same , no errors but output just for a bit of list . in dnsx , in httpx with asn flag same

dnsx -l URL.txt -asn -v
httpx ...

[Mzack9999]

memory should not increase in usage as the list is offloaded to disk via leveldb. I'm unable to replicate the crash. Would it be possible to share the target list? (feel free to join our discord and DM it to any pd team members referring this Github Issue or directly to me mzack9999). Thanks!