About urlscan.io

A sandbox for the web


urlscan.io is a service to scan and analyse websites. When a URL is submitted to urlscan.io, an automated process will browse to the URL like a regular user and record the activity that this page navigation creates. This includes the domains and IPs contacted, the resources (JavaScript, CSS, etc) requested from those domains, as well as additional information about the page itself. urlscan.io will take a screenshot of the page, record the DOM content, JavaScript global variables, cookies created by the page, and a myriad of other observations.

Finally, urlscan.io will try to make a verdict whether the scanned website is considered malicious or suspicious. If the site is targeting the users one of the almost 400 brands tracked by urlscan.io, this will be shown in the scan results.

You can contact us at info@urlscan.io. You can also use this GPG/PGP key to encrypt your mail.

Sponsors

We want to thank the following organizations for sponsoring urlscan.io, these folks help us to keep the lights on:

  • SecurityTrails - Security Data and APIs
    We Offer Paid API and Data Services for Top Security Companies. Tap into a treasure-trove of cyber security gold and get the info you can’t find anywhere else.
  • ipinfo.io - IP Address API and Data Solutions
    We're the trusted source for IP address data, handling 12 billion API requests per month for over 1,000 businesses and 100,000+ developers.
  • Tines - Security Automation and Orchestration (SOAR) Platform
    The Tines security automation platform helps the world's leading security teams automate any manual task. Making them more effective and efficient.

Commercial Products

We are offering a number of commercial options. All products offer a free time-limited trial period and can be purchased with a month-by-month subscription. Please contact us at sales@urlscan.io for more information and pricing.

Phishing URL Feed

urlscan.io detects more than 2000 malicious and phishing URLs targeting 400 popular brands every day. We are making the daily, weekly, and monthly feed available for commercial customers. The feed include the following pieces of information:

  • Phishing URL
  • Page title
  • Targeted Brand, Industry Vertical, Country of Origin
  • Domain & TLD of phishing URL
  • IP address hosting the phishing URL
  • GeoIP information hosting the phishing URL
  • ASN and ASN Name hosting the phishing URL
  • First-Seen Date of phishing URL
  • Country of submission
  • Aggregate information - Prevalence of brand, domain, IP, ASN
The feeds are available as CSV, JSON, TAXII, and STIX. Commercial use of the data is allowed.

urlscan Pro - Threat Hunting

The urlscan Pro System is a set of private APIs and data sources, coupled with a powerful new user interface. It operates on top of the publicly available data on urlscan.io. urlscan Pro supports a professional analyst by exposing more powerful query capabilities and pulling in more data to make sense of infrastructure and scanned websites. Users of urlscan Pro will have access to the following list of tools and resources. All features are available through the UI as well as a via a dedicated API.

  • Use a powerful search interface to hunt for interesting websites
  • Set alerts for specific keywords or infrastructure
  • Look at the scans detected as phishing by our phishing detection engine
  • Perform live investigations of suspicious websites from different geographical locations
  • Get abuse contact information and current site status for coordinating takedown requests

urlscan.io Sponsorship

If you are passionate users of urlscan.io and would like to support the public service at urlscan.io, consider becoming a sponsor! Sponsorship allows you to reach the roughly 70,000 daily unique users of the urlscan.io service. By showing your logo on urlscan.io, you are creating awareness and a positive image for your brand among the many information security professionals who use the service as part of their daily workflow.

  • Your logo on the front page of urlscan.io
  • Link from your logo and the "Sponsors" section to a website of your choosing
  • Frequent mention of your brand on the @urlscanio Twitter feed

Private Scan Plans Coming in 2020

In 2020, free users of urlscan.io will only be able to submit a limited amount of private scans per day. If you need to scan more websites you will be able to purchase different tiers of private submission volume.

  • Private scans in different tiers
  • Configurable data-retention period

urlscan On-Prem Coming in 2020

Some users of urlscan.io have legal constraints about the types of URLs they can submit to a public cloud-service. For these users we'll be offering a self-hosted on-prem version of urlscan.io which will include all of the features seen in our community platform.

  • API-enabled scanning appliance which can be used in standalone mode or integrated with a database and search
  • Scalalable scanning architecture
  • Captures HTTP requests, page screenshot, DOM snapshot, JavaScript variables, cookies and additional metadata
  • Search-index over previous scans
  • Optional per-scan settings such as user-agent, viewport size, timeouts, device emulation
  • Search across your own data as well as public scans from urlscan.io

FAQ

Q: How can I request the content of a scan to be removed from your website?
A: Please use the orange Report button on the result page of the scan.

Q: Can you prevent my domain from being scanned?
A: Yes, please send us a email at info@urlscan.io with the domains you want to be blacklisted.

Q: Does urlscan.io show whether a website contains malware or phishing attempts?
A: Yes, we have some basic mechanisms for determining whether a website contains malicious content. Our proprietary phishing detection mechanism tracks 400 popular brands and can identify phishing or impersonation attempts of these brands.
We do record file downloads, but we do not detect whether a downloaded file is malicious, e.g. a malicious executable.

Q: Can I search urlscan.io for pages which have been detected as malicious?
A: This feature is available as part of the urlscan Pro product and not available through the community search.

Q: Do you use my browser or Internet connection to analyze a website?
A: No! urlscan.io will browse any website you request itself, your browser is not involved. The website you want to scan will never learn your IP address and you will not be at risk when looking at the results.

Q: How does urlscan.io work?
A: We use the Google Chrome browser in Headless Mode to browse to the URLs submitted by users. We record the interaction of the page with the Internet and after the page has finished loading, we annotate the results with additional data sources.

Q: Do you store results indefinitely?
A: Yes, but right now we're not making a guarantee that the results of a scan will stay up for any period of time. When we hit certain limits we will have to start purging old scans.

Q: Do you support other browsers besides Google Chrome?
A: No, but you can set a custom User Agent during submission.

Q: Do you support IPv6?
A: Yes, and we're very happy about that because many similar services do not support it.
If you want to try a cool site, submit http://test-ipv6.com.

Q: Do private submissions deliver different results than public ones?
A: No, private submissions will deliver the same results as public ones. The only difference is that private submissions will not show up in the list of recently scanned sites and in the search results.

Q: Do you offer different browser locations/countries?
A: Not right now, we might include this feature in the future.

Q: Between different runs, websites often have a different number of HTTP transactions. Why is that?
A: The number of HTTP transactions depends on many factors:

  • Time of day and actual content of the site
  • Speed of the site (as we do have timeouts)
  • Advertising embedded in the site
Especially sites which employ a large amount of third-party ads and tracking will yield different results on each run. News website are the best example, as they often rely on ad revenue and have to deliver multiple ads from different sources.

Media Coverage

urlscan.io was covered by these posts, articles and screencasts:

  • securitytrails.com Blog - It's never been easier to make a great product: A chat with Johannes Gilger from urlscan.io (May 2, 2019)
  • tines.io - Automating abuse inbox management and phishing response (July 27, 2018)
  • The Daily Beast - Russian Hackers’ New Target: a Vulnerable Democratic Senator (July 26, 2018)
  • securitytrails.com Blog - URLScan.io: the best way to scan any website (July 16, 2018)

Reports referencing urlscan.io

These are industry reports that leverage urlscan.io or its data in some way.

Similar services & software

urlscan.io is not the only service that can be used to browse and analyse a website. These are some similar services, some provided invaluable inspiration for this very service!

Lists of similar & related services

Services

  • urlquery.net - Scans sites and looks up domains/IPs on various blacklists. This service inspired us to build urscan.io.
  • URLVoid - Website Reputation Checker Tool
  • keycdn speed test - Website speed test, employs similar techniques and inspired some features on this site
  • WebPagetest - Exhaustive speed-testing service with different locations, browser and options
  • pingdom Website speed test
  • Calibre Web performance monitoring - Professional service for monitoring web app performance
  • Trackography - Find out who is tracking you when you are reading your favourite news online.
  • Web Cookies Scanner - HTTP cookies, Flash, HTML5 localStorage, sessionStorage, CANVAS, supercookies, evercookies as well as SSL/TLS and HTTP security
  • Hardenize - Helping you deploy the latest security standards
  • Browserless - A headless browser in the cloud

Software

  • Lighthouse - analyzes web apps and web pages, collecting modern performance metrics and insights on developer best practices.
  • lightcrawler - Crawl a website and run it through Google lighthouse.
  • Puppeteer - Headless Chrome Node API, maintained by the Google Chrome Team
  • betwixt - System level network proxy, providing inspection via Network panel
  • Awesome chrome-devtools - Awesome tooling and resources in the Chrome DevTools ecosystem

Acknowledgements

Affiliation

urlscan.io is not affiliated with any of the services we link to on our results pages. Linking to any site does not constitute an endorsement or guarantee of fitness of the data.