urlscan.io Blog

We have just launched a suite of account security features and improvements to our core urlscan platform. Some of these features will benefit every user, while others are only available to commercial customers.

Active session control

You can now view and manage your active user sessions under the new Security tab in your user dashboard on urlscan.io. You will see your currently active sessions along with information about when and where these logged in from. Furthermore you have the ability to log-out (terminate) all other currently active sessions.

When you change your password, all other currently active sessions will automatically be terminated.

IP Whitelisting

You can limit access to your account to a fixed set of IP addresses and IP networks. This will apply both to interactive logins as well as API-initiated requests.

(This feature is only available as part of urlscan Enterprise and Ultimate)

Session lifetime settings

You are able to control the default idle session timeout as well as the maximum session age for your whole organisation. You can also choose to use so-called session cookies which are automatically cleared once your browser is closed.

Lastly, you can also activate exclusive sessions which will prevent a user within your organisation from logging in more than once at the same time.

(These features are only available as part of urlscan Enterprise and Ultimate)

Subscription notifications

For your team-account you can now add billing contacts as well as technical contacts. When these contacts are supplied we will be able to send you automated notifications when your current subscription is about to expire.

Additionally you will be able to observe a visual warning in your user dashboard when your subscription is nearing its renewal date.

Availability

These features are available today. If you want to make use of them please reach out to support@urlscan.io.

Today we are launching major improvements to our scanning engine and new scanning capabilities for our customers.

Live Browsing — Interact with websites

Our scanner has always been using the Google Chrome web-browser as the basis for its website analysis. Using Chrome allows the scanner to open and view websites exactly like a human user would. However, there are scenarios where additional human input is needed to access a website, such as dialogues or captchas.

With our new Live Browsing capability, customers on the urlscan Pro platform will be able to scan website while interacting with it through a VNC-like remote video and keyboard session. The primary use-case of Live Browsing is lightweight interaction, like dismissing alerts, confirming captchas or following a single level of redirection to get to the web-content of interest.

Live Browsing can also be used for other research-related tasks where the goal is not to create a scan result at the end. Some of these use-cases include:

• Capturing evidence for take-down purposes.
• Browsing through open directories.
• Browsing through the Tor network via .onion addresses.
• Quickly downloading files, DOM snapshots and screenshots from third-party websites.

Live Browsing is available today as a Beta feature for customers on the urlscan Pro platform.

Real Device Scanning

Some of our customers have encountered malicious websites which evade detection even when faced with the anti-detection methods available in our regular scanner. In some cases the only way to analyse these sites is to use an actual mobile device like an Android phone.

As part of the Live Browsing capability, we have also modified our scanning engine to analyze website using actual mobile devices, providing the high-fidelity output as our standard website scans.

Real Device Scanning is available to select Enterprise and Ultimate customers in Germany and the Netherlands for now. Reach out to support@urlscan.io if you are interest in this new capability.

Scanning Engine Improvements

In addition to the new features we have made various small improvements to our scanning engine and our Live Scanning UI in urlscan Pro:

• We have improved the device and user-agent selection in urlscan Pro.
• We have simplified the settings UI for Live Scans.
• We have improved the way language and locale settings are determined in the scanner.

Current state, availability

Live Browsing and Real Device Scanning are now available via the urlscan Pro platform. These features are currently in Beta, so there are no guarantees about their fitness for a particular purpose. Once these features are promoted to GA we will also implement certain rate limits according to applicable subscription plans.

If you want to learn about urlscan Pro platform and how it might be valuable for your organisation feel free to reach out to us! We offer free trial access without any strings attached. We would be happy to give you a passionate demo of what our platform can do for you. Reach out to us at sales@urlscan.io.

As we welcome the year 2024, we wanted to update you on what we have been working on in the second half of 2023 and announce the new features that are launching today. These changes will have a profound impact for our customer workflows and our own detection and classification abilities.

Saved Searches — A success story

When we launched Saved Searches in 2022 for our scans and hostnames feeds, we did not envision how popular this feature would turn out to be. Initially, Saved Searches were meant as a convenient way to bookmark a search term within the urlscan Pro platform. The Subscriptions feature allowed customers to receive notifications for any new items that matched their Saved Searches.

Over the past year, the value of Saved Searches to customers has become abundantly clear. Right now we manage more than 3000 Saved Searches and almost 1000 Subscriptions that have been created by our customers. Our subscription notification system sends out over 5000 emails a day.

Saved Searches and Subscriptions became even more important when we launched our Newly Observed Domains & Hostnames Feed in late 2022 and urlscan Observe earlier this year. Since then, many of our customers have set up Saved Searches to look for domains impersonating their brand or targeting their workforce. Our feed captures 2.5 million new domains and hostnames every day, so having an expressive search ability to find and alert on interesting hits is crucial.

Today we are launching major improvements for Saved Searches, Subscriptions and collaboration within the urlscan Pro platform.

→ Read the rest of this post...

urlscan.io has always been a powerful tool for scanning and investigating suspicious websites. Our platform is used by hundreds of customers and tens of thousands of community users to scan suspicious URLs. Up until now, the majority of these scans were initiated by customers.

Today we are announcing the general availability of urlscan Observe, our new and integrated hands-off monitoring system on the urlscan Pro platform. urlscan Observe ties together our extensive data collection with our notification and scanning features to drive fast and automated monitoring of suspected malicious infrastructure.

→ Read the rest of this post...

If you’re not sick of hearing it yet: Here’s to a happy new year from all of us at urlscan.io!

We wanted to take the opportunity to revisit major changes that launched in 2022 and to give you a glimpse of our 2023 roadmap at the same time. Some of the things we have worked on in 2022 represent the foundation for new products due to launch over the next quarters.

→ Read the rest of this post...

Today we are officially launching our real-time feed and search index of newly observed hostnames and domains on urlscan Pro. This is a huge step forward since it will allow customers to proactively look for new domains and hostnames that might be of interest to them, even if these hostnames were not previously scanned as a full-blown website through urlscan.io.

→ Read the rest of this post...

We are now offering raw download access to the following datasets to interested customers:

• Weekly Internet-wide scans of the whole IPv4 space on ports tcp/80 and tcp/443.
• JSON output containing TLS certificates and HTTP responses.
• More than 200GB of compressed raw data available per week.
• More than 40 million HTTP responses on tcp/80 and more than 35 million on tcp/443.

→ Read the rest of this post...

This post talks about the different scan visibilities available on urlscan.io, which visibility you should use for different purposes and how to review your submission results on urlscan.io to detect and prevent inadvertent information leaks.

tl;dr: Understand the different scan visibilities, review your own scans for non-public information, review your automated submission workflows, enforce a maximum scan visibility for your account and work with us to clean non-public data from urlscan.io!

→ Read the rest of this post...

Today marks the last day of major features releases we had planned for Q2. This post will cover the highlights of new functionality in our urlscan Pro platform.

Saved Searches & Subscriptions

You can now save a search in urlscan Pro to be able to run it again later. On top of that, you can also receive an email alert whenever there are new hits for your saved searches. This allows you to create a number of hunt queries which might only trigger occasionally and automatically receive notifications when there are new hits.

Search UI Improvements

The Search page in urlscan Pro was significantly improved:

• Filters are a convenient way to add, remove and invert pre-defined common search filters. We have a list of pre-defined filters that you can work with.
• In the Search view, the new Aggregations list shows you aggregate information from your search results and allows you to further filter your results by adding another facet to your filters.
• The new quick filter dialog also contains completions for the Brand Names that we track in urlscan Pro.
• Whenever you have created an interesting search, you can now save it as a Saved Search directly from the search UI.
• You can also use the new CSV Export feature to retrieve the results as a CSV file.

File Downloads

In the process of scanning websites, urlscan.io will sometimes encounter file downloads triggered by the website. If we are able to successfully download the file, we will store it, hash it and make it available for downloading by our customers.

To highlight this stream of data, we have created a separate Downloads section which contains the most recent file downloads and highlights the information we store for each downloaded file. There is a dedicated Help Section on Downloads which talks about API use and known limitations of this feature.

Live Scanning

The following features were added to the the Live Scanning UI:

• Additional devices available for device emulation (iPhone 12, 13, etc)
• Scanners can be selected via a new Select All button
• Scanner Details can be shown, such as the current exit IP, AS and VPN provider
• Scan Results have been cleaned up to give a better overview
• Outgoing Links can now be scanned with a dedicated button
• Available Live Scan Quotas are shown within the scanning UI

urlscan Pro Trial

If you would like to take urlscan Pro for a spin just reach out to sales@urlscan.io. We offer 30-day free trials with no strings attached.

As of today, our Live Scanning and Visual Search features are no longer considered Beta. The APIs for these features are now stable and can be used in production use-cases. Customers on our Professional and Enterprise subscription tiers will find API documentation for these features in the urlscan Pro platform.

Visual Search

Visual Search allows users to find historical scans with visually similar screenshots to a scan of interest. This type of feature is also called Content-Based Image Retrieval. Check out the accompanying blog-post to learn more.

Live Scanning

Live Scanning allows you to scan websites quickly, from different locations, and with different browser options. Scan results are not automatically saved to urlscan.io, but you can use Store Scan if you want to archive a particular scan result.

Live Scanning is a very versatile capability that can be used for a number of common scenarios, including Reconnaisance, Change Monitoring and Remote File Retrieval.

urlscan Pro Trial

If you would like to take urlscan Pro for a spin just reach out to sales@urlscan.io. We offer 30-day free trials with no strings attached.