Frequently Asked Questions
Data Sources
What data does Validin collect?
Validin collects the following types of data:
- DNS
- HTTP response headers and metadata
- Certificates from CT logs
- Registration Data
- OSINT Sources (1000+)
- Popularity Lists
- Block Lists
- Reputation Lists
- Malware Collections
Results
Why does NX sometimes overlap with other results?
Some DNS servers will respond with NXDOMAIN for labels that exist, but for which there are no records for the requested type. This is most common with AAAA (IPv6) records, but can happen with any record type. Validin requests A, AAAA, and NS records for each domain we track on at least a daily basis, so if we see NX for one query, but an answer for another, we’ll record both answers and display them when asked.
When do you display contiguous lines in your timelines vs. individual points?
Validin will combine data points into a single line if the time between consecutive measurements with the same value is less than 2 days.
Times appear to be aligned to 6-hour windows. Why?
We record DNS and banner answers with second granularity. However, to reduce storage space in our databases, we bucketize answers into 6-hour windows.
How often do you make DNS queries?
We make DNS queries throughout the day.
Where do you get your data?
We collect DNS and banner data in-house with custom crawling and scanning tooling and infrastructure. We use forward DNS with in-house closed recursive resolvers. By design, we do NOT monitor traffic from individuals and have no ability to see who is querying what at any level.
How do you determine what to query?
We query every name in our database at least daily. However, there are some names we drop from our database from time to time. For example, we only query the subdomains of some popular domains that are wildcarded (e.g., blogspot.com) when we see them explicitly referenced on a popularity or public block list.
Where do you get new names for forward DNS?
We discover new names from hundreds of independent and derivative sources, including popularity lists, zone files, DNS answers, and custom crawling results. We don’t have a reliance on any particular source for domain name discovery. We supplement this discovery process with educated guesses about other domains that might exist. For example, if google.