As we welcome the year 2024, we wanted to update you on what we have been
working on in the second half of 2023 and announce the new features that are
launching today. These changes will have a profound impact for our customer
workflows and our own detection and classification abilities.
Saved Searches — A success story
When we launched Saved Searches in 2022 for our scans
and hostnames
feeds, we did not envision how popular this feature would turn out to be.
Initially, Saved Searches were meant as a convenient way to bookmark a search
term within the urlscan Pro platform. The Subscriptions feature allowed
customers to receive notifications for any new items that matched their Saved
Searches.
Over the past year, the value of Saved Searches to customers has become
abundantly clear. Right now we manage more than 3000 Saved Searches and almost
1000 Subscriptions that have been created by our customers. Our subscription
notification system sends out over 5000 emails a day.
Saved Searches and Subscriptions became even more important when we launched
our Newly Observed Domains & Hostnames
Feed in late 2022 and
urlscan Observe earlier this
year. Since then, many of our customers have set up Saved Searches to look for
domains impersonating their brand or targeting their workforce. Our feed
captures 2.5 million new domains and hostnames every day, so having an
expressive search ability to find and alert on interesting hits is crucial.
Today we are launching major improvements for Saved Searches, Subscriptions and
collaboration within the urlscan Pro platform.
→ Read the rest of this post...
urlscan.io has always been a powerful tool for scanning and investigating
suspicious websites. Our platform is used by hundreds of customers and tens of
thousands of community users to scan suspicious URLs. Up until now, the
majority of these scans were initiated by customers.
Today we are announcing the general availability of urlscan Observe, our
new and integrated hands-off monitoring system on the urlscan Pro platform.
urlscan Observe ties together our extensive data collection with our
notification and scanning features to drive fast and automated monitoring of
suspected malicious infrastructure.
→ Read the rest of this post...
If you’re not sick of hearing it yet: Here’s to a happy new year from all of us
at urlscan.io!
We wanted to take the opportunity to revisit major changes that launched in
2022 and to give you a glimpse of our 2023 roadmap at the same time. Some of
the things we have worked on in 2022 represent the foundation for new products
due to launch over the next quarters.
→ Read the rest of this post...
Today we are officially launching our real-time feed and search index of newly
observed hostnames and domains on urlscan Pro. This is a huge step forward
since it will allow customers to proactively look for new domains and hostnames
that might be of interest to them, even if these hostnames were not previously
scanned as a full-blown website through urlscan.io.
→ Read the rest of this post...
We are now offering raw download access to the following datasets to interested customers:
- Weekly Internet-wide scans of the whole IPv4 space on ports tcp/80 and tcp/443.
- JSON output containing TLS certificates and HTTP responses.
- More than 200GB of compressed raw data available per week.
- More than 40 million HTTP responses on tcp/80 and more than 35 million on tcp/443.
→ Read the rest of this post...
This post talks about the different scan visibilities available on urlscan.io,
which visibility you should use for different purposes and how to review your
submission results on urlscan.io to detect and prevent inadvertent information
leaks.
tl;dr: Understand the different scan visibilities, review your own scans for non-public
information, review your automated submission workflows, enforce a maximum scan
visibility for your account and work with us to clean non-public data from urlscan.io!
→ Read the rest of this post...
Today marks the last day of major features releases we had planned for Q2. This
post will cover the highlights of new functionality in our urlscan Pro
platform.
Saved Searches & Subscriptions
You can now save a search in urlscan Pro to be able to run it again later. On
top of that, you can also receive an email alert whenever there are new hits
for your saved searches. This allows you to create a number of hunt queries
which might only trigger occasionally and automatically receive notifications
when there are new hits.
Search UI Improvements
The Search page in urlscan Pro was significantly improved:
- Filters are a convenient way to add, remove and invert pre-defined common
search filters. We have a list of pre-defined filters that you can work with.
- In the Search view, the new Aggregations list shows you aggregate
information from your search results and allows you to further filter your
results by adding another facet to your filters.
- The new quick filter dialog also contains completions for the Brand Names
that we track in urlscan Pro.
- Whenever you have created an interesting search, you can now save it as a
Saved Search directly from the search UI.
- You can also use the new CSV Export feature to retrieve the results as a
CSV file.
File Downloads
In the process of scanning websites, urlscan.io will sometimes encounter file
downloads triggered by the website. If we are able to successfully download the
file, we will store it, hash it and make it available for downloading by our
customers.
To highlight this stream of data, we have created a separate Downloads section
which contains the most recent file downloads and highlights the information we
store for each downloaded file. There is a dedicated Help Section on
Downloads which talks about API use and known limitations of this feature.
Live Scanning
The following features were added to the the Live Scanning UI:
- Additional devices available for device emulation (iPhone 12, 13, etc)
- Scanners can be selected via a new Select All button
- Scanner Details can be shown, such as the current exit IP, AS and VPN provider
- Scan Results have been cleaned up to give a better overview
- Outgoing Links can now be scanned with a dedicated button
- Available Live Scan Quotas are shown within the scanning UI
urlscan Pro Trial
If you would like to take urlscan Pro for a spin just reach out to
sales@urlscan.io. We offer 30-day free trials with
no strings attached.
As of today, our Live Scanning and Visual Search features are no longer
considered Beta. The APIs for these features are now stable and can be used
in production use-cases. Customers on our Professional and Enterprise
subscription tiers will find API documentation for these features in the
urlscan Pro platform.
Visual Search
Visual Search allows users to find historical scans with visually similar
screenshots to a scan of interest. This type of feature is also called
Content-Based Image Retrieval. Check out the accompanying
blog-post to learn more.
Live Scanning
Live Scanning allows you to scan websites quickly, from different locations,
and with different browser options. Scan results are not automatically saved to
urlscan.io, but you can use Store Scan if you want to archive a particular scan
result.
Live Scanning is a very versatile capability that can be used for a number of
common scenarios, including Reconnaisance, Change Monitoring and Remote
File Retrieval.
urlscan Pro Trial
If you would like to take urlscan Pro for a spin just reach out to
sales@urlscan.io. We offer 30-day free trials with
no strings attached.
Today we are launching Visual Search which is a powerful new search feature
available through our urlscan Pro - Threat Hunting platform.
Use-Cases
Visual Search allows users to find historical scans with visually similar
screenshots to a scan of interest. This type of feature is also called
Content-Based Image Retrieval. Instead of querying for historical scans using
a structured textual query (such as search for a hostname or an IP address),
Visual Search uses an existing screenshot image as the query. Visual Search
works similar to popular Reverse Image-Search engines like Google’s Search
by Image and the TinEye Reverse Image Search. Customers will be able to
leverage Visual Search feature to discover previously undetected cases of brand
impersonation or similar phishing pages based on the visual appearance of those
sites.
Availability
Visual Search is available today through the urlscan Pro portal. The feature is
currently in Beta until its API is finalized over the next few weeks. Further
information about Visual Search is available to customers on the urlscan Pro
platform.
urlscan Pro Trial
If you would like to take urlscan Pro for a spin just reach out to
sales@urlscan.io. We offer 30-day free trials with
no strings attached.
Today we are launching a major overhaul to our search index powering our
urlscan.io and urlscan Pro platforms. This release will offer new functionality
to community and paid users. We have gathered customer feedback and
internal use-cases and came up with a list of additional attributes that would
be helpful to search on. This post outlines the highlights of new available
search attributes. All of the new searchable fields have been
integrated in a backward compatible fashion, which means that any search
which previously worked on urlscan.io will continue to work.
The full list of searchable fields is available on the Search API
Reference page.
→ Read the rest of this post...