urlscan.io logo urlscan.io
SecurityTrails logo

www.webcrawler.com Open in urlscan Pro
34.241.67.236  Public Scan

Go To Rescan Add Verdict Report
Submitted URL: http://webcrawler.com/
Effective URL: http://www.webcrawler.com/
Submission: On April 02 via api from US
 Behaviour Indicators
Similar DOM Content API
Verdicts

Summary

This website contacted 7 IPs in 2 countries across 5 domains to perform 7 HTTP transactions. The main IP is 34.241.67.236, located in United States and belongs to AMAZON-02 - Amazon.com, Inc., US. The main domain is www.webcrawler.com.
This is the only time www.webcrawler.com was scanned on urlscan.io!

urlscan.io Verdict: No classification

Domain & IP information

IP Address AS Autonomous System
1 1 52.216.171.42 16509 (AMAZON-02)
1 34.241.67.236 16509 (AMAZON-02)
1 2a00:1450:400... 15169 (GOOGLE)
1 13.35.253.60 16509 (AMAZON-02)
2 2a00:1450:400... 15169 (GOOGLE)
1 13.35.253.124 16509 (AMAZON-02)
1 2600:9000:200... 16509 (AMAZON-02)
7 7
1    52.216.171.42 (Ashburn, United States)

ASN16509 (AMAZON-02 - Amazon.com, Inc., US)
PTR: s3-website-us-east-1.amazonaws.com
webcrawler.com
1    34.241.67.236 (United States)

ASN16509 (AMAZON-02 - Amazon.com, Inc., US)
PTR: ec2-34-241-67-236.eu-west-1.compute.amazonaws.com
www.webcrawler.com
1    2a00:1450:4001:820::2008 (Ireland)

ASN15169 (GOOGLE - Google LLC, US)
www.googletagmanager.com
1    13.35.253.60 (Seattle, United States)

ASN16509 (AMAZON-02 - Amazon.com, Inc., US)
PTR: server-13-35-253-60.fra6.r.cloudfront.net
s.flocdn.com
2    2a00:1450:4001:81e::200e (Ireland)

ASN15169 (GOOGLE - Google LLC, US)
www.google-analytics.com
1    13.35.253.124 (Seattle, United States)

ASN16509 (AMAZON-02 - Amazon.com, Inc., US)
PTR: server-13-35-253-124.fra6.r.cloudfront.net
s.flocdn.com
1    2600:9000:200c:3600:1:af78:4c0:93a1 (United States)

ASN16509 (AMAZON-02 - Amazon.com, Inc., US)
vendorlist.consensu.org
Domain Requested by
2 www.google-analytics.com www.googletagmanager.com
www.webcrawler.com
2 s.flocdn.com www.webcrawler.com
s.flocdn.com
1 vendorlist.consensu.org s.flocdn.com
1 www.googletagmanager.com www.webcrawler.com
1 www.webcrawler.com
1 webcrawler.com 1 redirects
7 6

This site contains links to these domains. Also see Links.

Domain
www.infospace.com
system1.com
Subject Issuer Validity Valid
*.google-analytics.com
Google Internet Authority G3		 2019-03-01 -
2019-05-24		 3 months crt.sh
*.flocdn.com
Amazon		 2018-05-31 -
2019-06-30		 a year crt.sh
vendorlist.consensu.org
Amazon		 2019-03-06 -
2020-04-06		 a year crt.sh

This page contains 1 frames:

Primary Page: http://www.webcrawler.com/
Frame ID: 7E7371EA341DE82A7CA6539DCD5AD401
Requests: 9 HTTP requests in this frame

Screenshot
Live screenshot
Full Image

Page URL History Show full URLs

  1. http://webcrawler.com/ HTTP 301
    http://www.webcrawler.com/ Page URL

Detected technologies

Overall confidence: 100%
Detected patterns
  • headers server /openresty(?:\/([\d.]+))?/i
Overall confidence: 100%
Detected patterns
  • headers server /openresty(?:\/([\d.]+))?/i
Overall confidence: 100%
Detected patterns
  • headers server /openresty(?:\/([\d.]+))?/i
Overall confidence: 100%
Detected patterns
  • script /google-analytics\.com\/(?:ga|urchin|(analytics))\.js/i
  • env /^gaGlobal$/i
Overall confidence: 100%
Detected patterns
  • env /^google_tag_manager$/i

Page Statistics

7
Requests

71 %
HTTPS

43 %
IPv6

5
Domains

6
Subdomains

7
IPs

2
Countries

109 kB
Transfer

362 kB
Size

3
Cookies

Page URL History

This captures the URL locations of the websites, including HTTP redirects and client-side redirects via JavaScript or Meta fields.

  1. http://webcrawler.com/ HTTP 301
    http://www.webcrawler.com/ Page URL

Redirected requests

There were HTTP redirect chains for the following requests:

7 HTTP transactions

Resource
Path		 Size
x-fer		 Type
MIME-Type
Primary Request /
www.webcrawler.com/
Redirect Chain
  • http://webcrawler.com/
  • http://www.webcrawler.com/
49 KB
18 KB 		Document
General
Check archive.org
Download Go to
Full URL
http://www.webcrawler.com/
Protocol
HTTP/1.1
Server
34.241.67.236 , United States, ASN16509 (AMAZON-02 - Amazon.com, Inc., US),
Reverse DNS
ec2-34-241-67-236.eu-west-1.compute.amazonaws.com
Software
openresty/1.11.2.2 /
Resource Hash
5acdd819ed6426d364f1eeb4bb24502ba48b9c9d29e905a43e1820bc9e9cbb5a

Request headers

Host
www.webcrawler.com
Connection
keep-alive
Pragma
no-cache
Cache-Control
no-cache
Upgrade-Insecure-Requests
1
User-Agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36
Accept
text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding
gzip, deflate
Upgrade-Insecure-Requests
1
User-Agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36

Response headers

Content-Encoding
gzip
Content-Type
text/html; charset=utf-8
Date
Tue, 02 Apr 2019 20:15:37 GMT
ETag
W/"4af8490018bfe7d03fa8d68ea9317c66"
Server
openresty/1.11.2.2
Vary
Accept-Encoding
transfer-encoding
chunked
Connection
keep-alive

Redirect headers

x-amz-id-2
REwJhlwMVFx8WypZZGHxaMo+DXkNc7yQVrNzK+3i90IFfHYQv2GP+PEDdbAKRbch+lF1VMdLdcY=
x-amz-request-id
3249C0413C53266B
Date
Tue, 02 Apr 2019 20:15:38 GMT
Location
http://www.webcrawler.com/
Content-Length
0
Server
AmazonS3
js
www.googletagmanager.com/gtag/ 		63 KB
24 KB 		Script
General
Check archive.org
Download Go to
Full URL
https://www.googletagmanager.com/gtag/js?id=UA-1764423-1
Requested by
Host: www.webcrawler.com
URL: http://www.webcrawler.com/
Protocol
H2
Security
TLS 1.2, ECDHE_RSA, AES_128_GCM
Server
2a00:1450:4001:820::2008 , Ireland, ASN15169 (GOOGLE - Google LLC, US),
Reverse DNS
Software
Google Tag Manager (scaffolding) /
Resource Hash
0f0453003b4c972e7e9661f7ecaf511c1b98d3377a618f13c5bfbbd9577b1de8
Security Headers
Name Value
X-Xss-Protection 0

Request headers

Referer
http://www.webcrawler.com/
User-Agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36

Response headers

date
Tue, 02 Apr 2019 20:15:37 GMT
content-encoding
br
last-modified
Tue, 02 Apr 2019 17:35:14 GMT
server
Google Tag Manager (scaffolding)
access-control-allow-origin
http://www.googletagmanager.com
vary
Accept-Encoding
content-type
application/javascript; charset=UTF-8
status
200
alt-svc
quic=":443"; ma=2592000; v="46,44,43,39"
cache-control
private, max-age=900
access-control-allow-credentials
true
access-control-allow-headers
Cache-Control
content-length
24777
x-xss-protection
0
expires
Tue, 02 Apr 2019 20:15:37 GMT
loader.js
s.flocdn.com/cmp/0.0.3-modal/ 		1 KB
1 KB 		Script
General
Check archive.org
Download Go to
Full URL
http://s.flocdn.com/cmp/0.0.3-modal/loader.js
Requested by
Host: www.webcrawler.com
URL: http://www.webcrawler.com/
Protocol
HTTP/1.1
Server
13.35.253.60 Seattle, United States, ASN16509 (AMAZON-02 - Amazon.com, Inc., US),
Reverse DNS
server-13-35-253-60.fra6.r.cloudfront.net
Software
AmazonS3 /
Resource Hash
49ac055a0e7961cf4a735167fc3a4bb07a259060a99ad02abcdeb0f2e178cfcb

Request headers

Referer
http://www.webcrawler.com/
User-Agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36

Response headers

x-amz-version-id
0iSURntPblqe.1C5FA2c4BfAqV6l89lD
Content-Encoding
gzip
Last-Modified
Fri, 02 Nov 2018 16:55:12 GMT
Server
AmazonS3
Age
73280
ETag
"d3b43b05dcbfa540a1eeb50b7278b8e4"
X-Cache
Hit from cloudfront
Content-Type
application/javascript; charset=UTF-8
Via
1.1 3095e870e1a1a1b03178e40ab1872de5.cloudfront.net (CloudFront)
Cache-Control
immutable
Date
Tue, 02 Apr 2019 19:07:17 GMT
Connection
keep-alive
Content-Length
597
X-Amz-Cf-Id
jNfkUPfIGMXmcn0cM6vjulKBjWysxCGLt5sswgOS_AwN81u5RSGhYg==
analytics.js
www.google-analytics.com/ 		43 KB
17 KB 		Script
General
Check archive.org
Download Go to
Full URL
https://www.google-analytics.com/analytics.js
Requested by
Host: www.googletagmanager.com
URL: https://www.googletagmanager.com/gtag/js?id=UA-1764423-1
Protocol
H2
Security
TLS 1.2, ECDHE_RSA, AES_128_GCM
Server
2a00:1450:4001:81e::200e , Ireland, ASN15169 (GOOGLE - Google LLC, US),
Reverse DNS
Software
Golfe2 /
Resource Hash
3e552578c7d450b023f2cd9d28f830be4335c3acc6c4ab6dadda0769f09e5f22
Security Headers
Name Value
Strict-Transport-Security max-age=10886400; includeSubDomains; preload
X-Content-Type-Options nosniff

Request headers

Referer
http://www.webcrawler.com/
User-Agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36

Response headers

strict-transport-security
max-age=10886400; includeSubDomains; preload
content-encoding
gzip
x-content-type-options
nosniff
last-modified
Wed, 16 Jan 2019 20:01:45 GMT
server
Golfe2
age
1721
date
Tue, 02 Apr 2019 19:46:56 GMT
vary
Accept-Encoding
content-type
text/javascript
status
200
cache-control
public, max-age=7200
timing-allow-origin
*
alt-svc
quic=":443"; ma=2592000; v="46,44,43,39"
content-length
17543
expires
Tue, 02 Apr 2019 21:46:56 GMT
truncated
/ 		8 KB
0 		Image
General
Check archive.org
Download Go to Show image
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
838c9fc0eed0aadff0e091429f4ccce2c9a1ee5361224ccd6ac05ed484bdc16b

Request headers

Referer
http://www.webcrawler.com/
User-Agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36

Response headers

Content-Type
image/svg+xml;charset=utf-8
truncated
/ 		466 B
0 		Image
General
Check archive.org
Download Go to Show image
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
3d9ba870d6ae668ce310b0905ec24e74b9cc44cd4d80b248ae5f027b7b5eaecb

Request headers

Referer
http://www.webcrawler.com/
User-Agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36

Response headers

Content-Type
image/svg+xml;charset=utf-8
collect
www.google-analytics.com/r/ 		35 B
111 B 		Image
General
Check archive.org
Download Go to Show image
Full URL
https://www.google-analytics.com/r/collect?v=1&_v=j73&a=2138783284&t=pageview&_s=1&dl=http%3A%2F%2Fwww.webcrawler.com%2F&ul=en-us&de=UTF-8&dt=WebCrawler%20Search&sd=24-bit&sr=1600x1200&vp=1600x1200&je=0&_u=IEBAAUAB~&jid=307223933&gjid=477033954&cid=1010935511.1554236138&tid=UA-1764423-1&_gid=1504774442.1554236138&_r=1&gtm=2ou3i1&z=1797932684
Requested by
Host: www.webcrawler.com
URL: http://www.webcrawler.com/
Protocol
H2
Security
TLS 1.2, ECDHE_RSA, AES_128_GCM
Server
2a00:1450:4001:81e::200e , Ireland, ASN15169 (GOOGLE - Google LLC, US),
Reverse DNS
Software
Golfe2 /
Resource Hash
8337212354871836e6763a41e615916c89bac5b3f1f0adf60ba43c7c806e1015
Security Headers
Name Value
X-Content-Type-Options nosniff

Request headers

Referer
http://www.webcrawler.com/
User-Agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36

Response headers

pragma
no-cache
date
Tue, 02 Apr 2019 20:15:37 GMT
x-content-type-options
nosniff
last-modified
Sun, 17 May 1998 03:00:00 GMT
server
Golfe2
access-control-allow-origin
*
content-type
image/gif
status
200
cache-control
no-cache, no-store, must-revalidate
alt-svc
quic=":443"; ma=2592000; v="46,44,43,39"
content-length
35
expires
Fri, 01 Jan 1990 00:00:00 GMT
cmp.js
s.flocdn.com/cmp/0.0.3-modal/ 		121 KB
33 KB 		Script
General
Check archive.org
Download Go to
Full URL
https://s.flocdn.com/cmp/0.0.3-modal/cmp.js
Requested by
Host: s.flocdn.com
URL: http://s.flocdn.com/cmp/0.0.3-modal/loader.js
Protocol
H2
Security
TLS 1.2, ECDHE_RSA, AES_128_GCM
Server
13.35.253.124 Seattle, United States, ASN16509 (AMAZON-02 - Amazon.com, Inc., US),
Reverse DNS
server-13-35-253-124.fra6.r.cloudfront.net
Software
AmazonS3 /
Resource Hash
7c05dced1ff0baacf48eef0c03507348ee9dd55530af4911a4957c54018c690c

Request headers

Referer
http://www.webcrawler.com/
User-Agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36

Response headers

x-amz-version-id
5HNG9REzZyRoM.bFhxhsW9N5BtxYGSRw
content-encoding
gzip
last-modified
Fri, 02 Nov 2018 16:55:12 GMT
server
AmazonS3
age
74147
etag
"b71bb86c5de6b316396d7f836ce6c667"
x-cache
Hit from cloudfront
content-type
application/javascript; charset=UTF-8
status
200
cache-control
immutable
date
Tue, 02 Apr 2019 19:07:18 GMT
content-length
33699
via
1.1 49140b838a62cd29e30f20e39a82dad0.cloudfront.net (CloudFront)
x-amz-cf-id
n6T6aZ1R7s48wSL8nTc3bAjoTh1T1c_vG_IRow10-QO70M-jmtRReQ==
vendorlist.json
vendorlist.consensu.org/ 		77 KB
15 KB 		Fetch
General
Check archive.org
Download Go to
Full URL
https://vendorlist.consensu.org/vendorlist.json
Requested by
Host: s.flocdn.com
URL: https://s.flocdn.com/cmp/0.0.3-modal/cmp.js
Protocol
H2
Security
TLS 1.2, ECDHE_RSA, AES_128_GCM
Server
2600:9000:200c:3600:1:af78:4c0:93a1 , United States, ASN16509 (AMAZON-02 - Amazon.com, Inc., US),
Reverse DNS
Software
AmazonS3 /
Resource Hash
7c7b42ca335ca1d12d8e231ae1badc264ebf8e079a23b9f8ff5beee110e5640b

Request headers

User-Agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36
Referer
http://www.webcrawler.com/
Origin
http://www.webcrawler.com

Response headers

date
Fri, 29 Mar 2019 08:32:34 GMT
content-encoding
gzip
vary
Accept-Encoding,Origin,Access-Control-Request-Headers,Access-Control-Request-Method
age
40752
x-cache
Hit from cloudfront
status
200
access-control-allow-origin
*
last-modified
Thu, 28 Mar 2019 16:00:19 GMT
server
AmazonS3
access-control-max-age
86400
access-control-allow-methods
GET
x-amz-version-id
Y4gcMt2wVaEtlgHWiE_EYPxW_yGStyal
via
1.1 60a935292c9892b0b7f9e56f65af863a.cloudfront.net (CloudFront)
cache-control
max-age=259200
content-type
application/json; charset=utf-8
x-amz-cf-id
Sg5cHG91NNFjyeoO3a48bzqJUCxXhGye4CcqqqnaLMcEUDCMnZcA5g==

Verdicts & Comments Add Verdict or Comment

27 JavaScript Global Variables

These are the non-standard "global" variables defined on the window object. These can be helpful in identifying possible client-side frameworks and code.

object| onselectstart object| onselectionchange function| queueMicrotask object| __core-js_shared__ boolean| isIE11 function| debounce function| updateUrlParameter function| getUrlParameterByName function| removeUrlParameter function| smoothScroll function| createPostForm function| updateSearchPreferences function| hide function| show object| dataLayer object| google_tag_manager string| GoogleAnalyticsObject function| ga object| google_tag_data object| gaplugins object| gaGlobal object| gaData function| cmp function| __cmp string| onConsentChangedUrl function| showConsentTool object| core

3 Cookies

Domain/Path Name / Value
.webcrawler.com/ Name: _gat_gtag_UA_1764423_1
Value: 1
.webcrawler.com/ Name: _gid
Value: GA1.2.1504774442.1554236138
.webcrawler.com/ Name: _ga
Value: GA1.2.1010935511.1554236138