Internet-Wide IPv4 Scan Data
We are now offering raw download access to the following datasets to interested customers:
- Weekly Internet-wide scans of the whole IPv4 space on ports tcp/80 and tcp/443.
- JSON output containing TLS certificates and HTTP responses.
- More than 200GB of compressed raw data available per week.
- More than 40 million HTTP responses on tcp/80 and more than 35 million on tcp/443.
The HTTP scan results contain the following pieces of data and metadata:
- Observation timestamp
- GeoIP and ASN network annotation for the IP which was scanned
- HTTP Status code & string
- HTTP Response Headers & content
- Response Body Size & SHA256
- Response Body (up to 64kB)
- TLS Certificate Subject, Issuer, Serial Number and MD5/SHA1/SHA256 fingerprints
The certificate scan results contain certificates that responded to HTTPS requests:
- Raw unparsed TLS certificate data
- Parsed certificate data
Sample Data
These samples are 10,000 lines from the respective datasets and should give you an idea what’s contained within the data.
HTTP Scans on tcp/80 - HTTP Responses
HTTP Scans on tcp/443 - HTTP Responses
HTTP Scans on tcp/443 - TLS Certificates
Access to Data
If you are interested in access to these data-sets then please reach out to us at support@urlscan.io. We offer free 30-day trial access.