www.digitalocean.com Open in urlscan Pro
104.16.25.4  Public Scan

16 Outgoing links

These are links going to different origins than the main page.

Search URL Search Domain Scan URL

---

Search URL Search Domain Scan URL

---

Search URL Search Domain Scan URL

---

Search URL Search Domain Scan URL

---

Search URL Search Domain Scan URL

---

Search URL Search Domain Scan URL

---

Search URL Search Domain Scan URL

---

Search URL Search Domain Scan URL

---

Search URL Search Domain Scan URL

---

Search URL Search Domain Scan URL

---

Search URL Search Domain Scan URL

---

Search URL Search Domain Scan URL

---

Search URL Search Domain Scan URL

---

Search URL Search Domain Scan URL

---

Search URL Search Domain Scan URL

---

Search URL Search Domain Scan URL

---

Redirected requests

There were HTTP redirect chains for the following requests:

Request Chain 37

• https://www.facebook.com/connect/ping?client_id=694818843983011&domain=www.digitalocean.com&origin=1&redirect_uri=https%3A%2F%2Fstaticxx.facebook.com%2Fconnect%2Fxd_arbiter%2Fr%2FhsBwMj6iLmk.js%3Fversion%3D42%23cb%3Df36de03d2b1e89c%26domain%3Dwww.digitalocean.com%26origin%3Dhttps%253A%252F%252Fwww.digitalocean.com%252Ff2eec6df4d77994%26relation%3Dparent&response_type=token%2Csigned_request%2Ccode&sdk=joey HTTP 302
• https://staticxx.facebook.com/connect/xd_arbiter/r/hsBwMj6iLmk.js?version=42

Request Chain 43

• https://www.google-analytics.com/r/collect?v=1&_v=j64&a=174397586&t=pageview&_s=1&dl=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3&ul=en-us&de=UTF-8&dt=Crawling%20and%20Scraping%20Web%20Pages%20with%20Scrapy%20and%20Python%203%20%7C%20DigitalOcean&sd=24-bit&sr=1600x1200&vp=1585x1200&je=0&_u=aGBAAAAjI~&jid=563438466&gjid=207446299&cid=1665569122.1508777325&tid=UA-26573244-1&_gid=738995134.1508777325&_r=1&gtm=GajKHWBBT&z=171347024 HTTP 302
• https://stats.g.doubleclick.net/r/collect?v=1&aip=1&t=dc&_r=3&tid=UA-26573244-1&cid=1665569122.1508777325&jid=563438466&_gid=738995134.1508777325&gjid=207446299&_v=j64&z=171347024 HTTP 302
• https://www.google.com/ads/ga-audiences?v=1&aip=1&t=sr&_r=4&tid=UA-26573244-1&cid=1665569122.1508777325&jid=563438466&_v=j64&z=171347024 HTTP 302
• https://www.google.de/ads/ga-audiences?v=1&aip=1&t=sr&_r=4&tid=UA-26573244-1&cid=1665569122.1508777325&jid=563438466&_v=j64&z=171347024&slf_rd=1&random=2039696993

Request Chain 46

• https://d.company-target.com/pixel?type=js&id=1501520880&page=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3 HTTP 302
• https://d.company-target.com/ul_cb/pixel?type=js&id=1501520880&page=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3

Request Chain 47

• https://d.company-target.com/pixel?type=js&id=1501520919&page=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3 HTTP 302
• https://d.company-target.com/ul_cb/pixel?type=js&id=1501520919&page=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3

50 HTTP transactions Everything HTML Script AJAX CSS Image Expand all
0 data transactions

Method Protocol	Status	Resource Path	Size x-fer	Time Latency	Type MIME-Type	IP Location
GET H2	200	Primary Request how-to-crawl-a-web-page-with-scrapy-and-python-3 Show response www.digitalocean.com/community/tutorials/	71 KB 21 KB	2245ms 2224ms	Document text/html	104.16.25.4 CloudFlare
General Check archive.org Show headers Download Go to Full URL https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 104.16.25.4 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash 1347b8d186dc0350ac80780e84ff375f660638f7e9b6b6c8b44e08d23c706eff Security Headers Name Value X-Content-Type-Options nosniff X-Frame-Options SAMEORIGIN X-Xss-Protection 1; mode=block Request headers :path /community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 pragma no-cache accept-encoding gzip, deflate upgrade-insecure-requests 1 user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8 cache-control no-cache :authority www.digitalocean.com :scheme https :method GET Upgrade-Insecure-Requests 1 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers x-runtime 1.621323 date Mon, 23 Oct 2017 16:48:43 GMT content-encoding gzip x-content-type-options nosniff server cloudflare-nginx x-frame-options SAMEORIGIN content-type text/html; charset=utf-8 status 200 cache-control max-age=0, private, must-revalidate set-cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; expires=Tue, 23-Oct-18 16:48:40 GMT; path=/; domain=.digitalocean.com; HttpOnly first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; domain=.digitalocean.com; path=/; expires=Tue, 23 Jan 2018 16:48:41 -0000 last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; domain=.digitalocean.com; path=/ referrer=; domain=.digitalocean.com; path=/; expires=Tue, 23 Jan 2018 16:48:41 -0000 _community_session=MGVndnNtNy9URy9TUW9vNm5CL0ZTQzBNajZuM2RRRHBMckFSVGpxcmd0TS9oSml0Z1RxQTNkbUdmeWV5YldRVDJIMENhbjdJZTRmaEkzUVRGR3hTdlFqaXlBZERJcGw1R2I0OXN6VnQycUNmMUxkZ2VBU3cvV3orTzQwck5hdFBTYTZtckJvYXB3VlB1MHZaVXNMNk01SjR2ckFoN3I0MjBHcWVxYW1LNHlheG1ITHhydmpyWXUwbGF2UXZuKzlNQXl1bUhNMEtLbEJ4a3FzcUV0cmJLR3NGZUdrOHBqWUpWZElnVVNIUVN2d3lyVzZDMTQ0R215aHBKdFpHYVRaUkdJVnc5QmRVVFozYTRReUErbmVTb2RXbWhXT3BrKzNMclY5MW5Td3Y0Uyt4bnIrVTRZbTZHelo1SE9jK1hxSVkwTzBlOGxBWXByUVIvSEM1eEthTmpBPT0tLTl3aGs1ZjNWUS8vTGhrTWVTRkVsSXc9PQ%3D%3D--a15aad9bbe5e226c0ee49dd0039533402d1085ae; path=/; HttpOnly cf-ray 3b262f701f5426ea-FRA x-xss-protection 1; mode=block x-request-id 3eb3bd26-5305-4e1c-a731-18490bff752a
GET H2	200	application-59796a7d20124872c204c9bc8c7193ef.css www.digitalocean.com/assets/community/	396 KB 64 KB	15ms 13ms	Stylesheet text/css	104.16.25.4 CloudFlare
General Check archive.org Show headers Download Go to Full URL https://www.digitalocean.com/assets/community/application-59796a7d20124872c204c9bc8c7193ef.css Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 104.16.25.4 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash 826c988aa9929c5522e13cd799a521504ef89f7fa292468dc7d8ec9da612c73d Request headers :path /assets/community/application-59796a7d20124872c204c9bc8c7193ef.css pragma no-cache cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; referrer=; _community_session=MGVndnNtNy9URy9TUW9vNm5CL0ZTQzBNajZuM2RRRHBMckFSVGpxcmd0TS9oSml0Z1RxQTNkbUdmeWV5YldRVDJIMENhbjdJZTRmaEkzUVRGR3hTdlFqaXlBZERJcGw1R2I0OXN6VnQycUNmMUxkZ2VBU3cvV3orTzQwck5hdFBTYTZtckJvYXB3VlB1MHZaVXNMNk01SjR2ckFoN3I0MjBHcWVxYW1LNHlheG1ITHhydmpyWXUwbGF2UXZuKzlNQXl1bUhNMEtLbEJ4a3FzcUV0cmJLR3NGZUdrOHBqWUpWZElnVVNIUVN2d3lyVzZDMTQ0R215aHBKdFpHYVRaUkdJVnc5QmRVVFozYTRReUErbmVTb2RXbWhXT3BrKzNMclY5MW5Td3Y0Uyt4bnIrVTRZbTZHelo1SE9jK1hxSVkwTzBlOGxBWXByUVIvSEM1eEthTmpBPT0tLTl3aGs1ZjNWUS8vTGhrTWVTRkVsSXc9PQ%3D%3D--a15aad9bbe5e226c0ee49dd0039533402d1085ae accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept text/css,*/*;q=0.1 cache-control no-cache :authority www.digitalocean.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:43 GMT content-encoding gzip cf-cache-status HIT last-modified Wed, 04 Oct 2017 17:57:28 GMT server cloudflare-nginx etag "59d52108-fe38" vary Accept-Encoding content-type text/css status 200 cache-control public, max-age=315360000 cf-ray 3b262f7e0f3c26ea-FRA content-length 65080 expires Thu, 21 Oct 2027 16:48:43 GMT
GET H2	200	izu1uqu.js Show response use.typekit.net/	18 KB 7 KB	23ms 7ms	Script text/javascript	92.123.94.15 AKAMAI-ASN1
General Check archive.org Show headers Download Go to Full URL https://use.typekit.net/izu1uqu.js Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_256_GCM Server 92.123.94.15 , European Union, ASN20940 (AKAMAI-ASN1, US), Reverse DNS a92-123-94-15.deploy.akamaitechnologies.com Software nginx / Resource Hash b674207b25e36c72ddf3cc0d9234e8d72c36f0c2fd1e432aada6412dafa8c4ec Security Headers Name Value Strict-Transport-Security max-age=31536000; includeSubDomains; Request headers :path /izu1uqu.js pragma no-cache accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority use.typekit.net referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers strict-transport-security max-age=31536000; includeSubDomains; content-encoding gzip server nginx status 200 200 OK date Mon, 23 Oct 2017 16:48:43 GMT vary Accept-Encoding content-type text/javascript;charset=utf-8 access-control-allow-origin * cache-control public, max-age=604800 timing-allow-origin * content-length 7060
GET H/1.1	200 OK	bugsnag-2.min.js Show response d2wy8f7a9ursnm.cloudfront.net/	6 KB 3 KB	15ms 13ms	Script application/javascript	54.230.15.15 Amazon.com
General Check archive.org Show headers Download Go to Full URL https://d2wy8f7a9ursnm.cloudfront.net/bugsnag-2.min.js Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol HTTP/1.1 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 54.230.15.15 Seattle, United States, ASN16509 (AMAZON-02 - Amazon.com, Inc., US), Reverse DNS server-54-230-15-15.ams1.r.cloudfront.net Software AmazonS3 / Resource Hash 9ff538f72465724fc393ea1f3c03a17233c9b7e1d440d6f8a6d0b3a836c2a9cc Request headers Pragma no-cache Accept-Encoding gzip, deflate Host d2wy8f7a9ursnm.cloudfront.net User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Accept */* Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Connection keep-alive Cache-Control no-cache Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers Date Mon, 12 Jun 2017 03:14:01 GMT Content-Encoding gzip Last-Modified Wed, 10 Aug 2016 00:30:49 GMT Server AmazonS3 Age 46089 ETag "6103bb5e4ec6141e19e1100caafc780c" X-Cache Hit from cloudfront Content-Type application/javascript Via 1.1 fb6cb783855196b3edbc2c1ca52f74d0.cloudfront.net (CloudFront) Cache-Control public, max-age=604800 Connection keep-alive Accept-Ranges bytes Content-Length 2962 X-Amz-Cf-Id xx5KvRCB4DtEc0euqYbRzfY-kw6b_BFW5MldMpW1DdANaM2u7Ve5ww==
GET H2	200	polyfill.min.js Show response cdn.polyfill.io/v2/	72 B 99 B	38ms 9ms	Script application/javascript	2a04:4e42:1b::621 Fastly
General Check archive.org Show headers Download Go to Full URL https://cdn.polyfill.io/v2/polyfill.min.js Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 2a04:4e42:1b::621 , European Union, ASN54113 (FASTLY - Fastly, US), Reverse DNS Software Cowboy / Resource Hash aaecd144d2b8763b2fa5c91f09778294363cef363c10504205f4203922644d11 Security Headers Name Value Strict-Transport-Security max-age=31536000; includeSubdomains; preload X-Content-Type-Options nosniff X-Frame-Options sameorigin X-Xss-Protection 1; mode=block Request headers :path /v2/polyfill.min.js pragma no-cache accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority cdn.polyfill.io referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:43 GMT content-encoding gzip x-content-type-options nosniff age 0 x-cache HIT status 200 vary Accept-Encoding User-Agent content-length 90 x-xss-protection 1; mode=block x-served-by cache-hhn1550-HHN access-control-allow-origin * server Cowboy x-timer S1508777323.230943,VS0,VE1 x-frame-options sameorigin strict-transport-security max-age=31536000; includeSubdomains; preload content-type application/javascript;charset=utf-8 via 1.1 vegur 1.1 varnish cache-control public, s-maxage=31536000, max-age=604800, stale-while-revalidate=604800, stale-if-error=604800 accept-ranges bytes timing-allow-origin * x-cache-hits 1
GET H2	200	forms2.min.js Show response go.digitalocean.com/js/forms2/js/	165 KB 56 KB	95ms 64ms	Script application/x-javascript	104.16.111.208 CloudFlare
General Check archive.org Show headers Download Go to Full URL https://go.digitalocean.com/js/forms2/js/forms2.min.js Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol H2 Security TLS 1.2, ECDHE_ECDSA, AES_128_GCM Server 104.16.111.208 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash aaee78be73219813ee518842197fffc34bc09d755f52f4e829fd8ffec460f876 Security Headers Name Value X-Content-Type-Options nosniff Request headers :path /js/forms2/js/forms2.min.js pragma no-cache cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; referrer= accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority go.digitalocean.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:43 GMT content-encoding gzip x-content-type-options nosniff cf-cache-status HIT last-modified Mon, 25 Sep 2017 19:07:26 GMT server cloudflare-nginx etag "32039b-292eb-55a0844ea5780" vary Accept-Encoding content-type application/x-javascript status 200 cache-control public, max-age=7200 cf-ray 3b262f7e3e5b63a9-FRA expires Mon, 23 Oct 2017 18:48:43 GMT
GET H2	200	application-ed83ed45704e8c3619c0698b180fdc77.js Show response www.digitalocean.com/assets/community/	611 KB 184 KB	20ms 19ms	Script application/x-javascript	104.16.25.4 CloudFlare
General Check archive.org Show headers Download Go to Full URL https://www.digitalocean.com/assets/community/application-ed83ed45704e8c3619c0698b180fdc77.js Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 104.16.25.4 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash 8a50c603fbb1d79bfae1f70ea9c0d95de0abc9c7be012e8d6912bb2b1a028ca0 Request headers :path /assets/community/application-ed83ed45704e8c3619c0698b180fdc77.js pragma no-cache cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; referrer=; _community_session=MGVndnNtNy9URy9TUW9vNm5CL0ZTQzBNajZuM2RRRHBMckFSVGpxcmd0TS9oSml0Z1RxQTNkbUdmeWV5YldRVDJIMENhbjdJZTRmaEkzUVRGR3hTdlFqaXlBZERJcGw1R2I0OXN6VnQycUNmMUxkZ2VBU3cvV3orTzQwck5hdFBTYTZtckJvYXB3VlB1MHZaVXNMNk01SjR2ckFoN3I0MjBHcWVxYW1LNHlheG1ITHhydmpyWXUwbGF2UXZuKzlNQXl1bUhNMEtLbEJ4a3FzcUV0cmJLR3NGZUdrOHBqWUpWZElnVVNIUVN2d3lyVzZDMTQ0R215aHBKdFpHYVRaUkdJVnc5QmRVVFozYTRReUErbmVTb2RXbWhXT3BrKzNMclY5MW5Td3Y0Uyt4bnIrVTRZbTZHelo1SE9jK1hxSVkwTzBlOGxBWXByUVIvSEM1eEthTmpBPT0tLTl3aGs1ZjNWUS8vTGhrTWVTRkVsSXc9PQ%3D%3D--a15aad9bbe5e226c0ee49dd0039533402d1085ae accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority www.digitalocean.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:43 GMT content-encoding gzip cf-cache-status HIT last-modified Thu, 31 Aug 2017 18:24:21 GMT server cloudflare-nginx etag "59a85455-2e02a" vary Accept-Encoding content-type application/x-javascript status 200 cache-control public, max-age=315360000 cf-ray 3b262f7e0f3d26ea-FRA content-length 188458 expires Thu, 21 Oct 2027 16:48:43 GMT
GET H2	200	d753bf40d7283f16ec0db2c9b0d18b58 secure.gravatar.com/avatar/	3 KB 3 KB	18ms 6ms	Image image/jpeg	192.0.73.2 Automattic
General Check archive.org Show headers Download Go to Show image Full URL https://secure.gravatar.com/avatar/d753bf40d7283f16ec0db2c9b0d18b58?secure=true&d=identicon Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 192.0.73.2 San Francisco, United States, ASN2635 (AUTOMATTIC - Automattic, Inc, US), Reverse DNS Software nginx / Resource Hash d34f8f1e9654a2e36baee16bab4b40af259923ac0d1ef86784e1ff50be226c9f Request headers :path /avatar/d753bf40d7283f16ec0db2c9b0d18b58?secure=true&d=identicon pragma no-cache accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept image/webp,image/apng,image/*,*/*;q=0.8 cache-control no-cache :authority secure.gravatar.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers x-nc HIT fra 3 date Mon, 23 Oct 2017 16:48:43 GMT last-modified Sat, 07 Jan 2017 20:46:12 GMT server nginx source-age 15797 status 200 content-type image/jpeg access-control-allow-origin * cache-control max-age=300 content-disposition inline; filename="d753bf40d7283f16ec0db2c9b0d18b58.jpeg" accept-ranges bytes link <https://www.gravatar.com/avatar/d753bf40d7283f16ec0db2c9b0d18b58?secure=true&d=identicon>; rel="canonical" content-length 2850 expires Mon, 23 Oct 2017 16:53:43 GMT
GET H2	200	creativecommons-08b32a9279fcd47fcd78ac6a26331389.png www.digitalocean.com/assets/community/	1 KB 1 KB	16ms 16ms	Image image/png	104.16.25.4 CloudFlare
General Check archive.org Show headers Download Go to Show image Full URL https://www.digitalocean.com/assets/community/creativecommons-08b32a9279fcd47fcd78ac6a26331389.png Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 104.16.25.4 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash 027bb7f065acf05ba3c0f84a040d2da641648afc81daa6ff5570301d4998bbb6 Request headers :path /assets/community/creativecommons-08b32a9279fcd47fcd78ac6a26331389.png pragma no-cache cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; referrer=; _community_session=MGVndnNtNy9URy9TUW9vNm5CL0ZTQzBNajZuM2RRRHBMckFSVGpxcmd0TS9oSml0Z1RxQTNkbUdmeWV5YldRVDJIMENhbjdJZTRmaEkzUVRGR3hTdlFqaXlBZERJcGw1R2I0OXN6VnQycUNmMUxkZ2VBU3cvV3orTzQwck5hdFBTYTZtckJvYXB3VlB1MHZaVXNMNk01SjR2ckFoN3I0MjBHcWVxYW1LNHlheG1ITHhydmpyWXUwbGF2UXZuKzlNQXl1bUhNMEtLbEJ4a3FzcUV0cmJLR3NGZUdrOHBqWUpWZElnVVNIUVN2d3lyVzZDMTQ0R215aHBKdFpHYVRaUkdJVnc5QmRVVFozYTRReUErbmVTb2RXbWhXT3BrKzNMclY5MW5Td3Y0Uyt4bnIrVTRZbTZHelo1SE9jK1hxSVkwTzBlOGxBWXByUVIvSEM1eEthTmpBPT0tLTl3aGs1ZjNWUS8vTGhrTWVTRkVsSXc9PQ%3D%3D--a15aad9bbe5e226c0ee49dd0039533402d1085ae accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept image/webp,image/apng,image/*,*/*;q=0.8 cache-control no-cache :authority www.digitalocean.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:43 GMT cf-cache-status HIT last-modified Mon, 23 May 2016 14:48:37 GMT server cloudflare-nginx etag "57431845-419" vary Accept-Encoding content-type image/png status 200 cache-control public, max-age=315360000 accept-ranges bytes cf-ray 3b262f7e0f4126ea-FRA content-length 1049 expires Thu, 21 Oct 2027 16:48:43 GMT
GET H2	200	analytics.min.js Show response cdn.segment.com/analytics.js/v1/puo3uv968t/	233 KB 53 KB	44ms 15ms	Script text/javascript	54.230.15.57 Amazon.com
General Check archive.org Show headers Download Go to Full URL https://cdn.segment.com/analytics.js/v1/puo3uv968t/analytics.min.js Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 54.230.15.57 Seattle, United States, ASN16509 (AMAZON-02 - Amazon.com, Inc., US), Reverse DNS server-54-230-15-57.ams1.r.cloudfront.net Software nginx / Resource Hash fb6ebe00de55ee51f18a612a71e8e584357a6f2a1b11a6f17cf5bcf55d48dd88 Request headers :path /analytics.js/v1/puo3uv968t/analytics.min.js pragma no-cache accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority cdn.segment.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 14:40:47 GMT content-encoding gzip server nginx age 89 status 200 etag W/"3a591-kyUDnOVPl2wpe3/kuuATDA" x-cache-status HIT vary Accept-Encoding x-cache Hit from cloudfront content-type text/javascript; charset=utf-8 access-control-allow-origin * cache-control public, max-age=120 x-amz-cf-id PBmhhs3QYBkj91tE5EM1imL41G9v22QypjxQoLOnvoF3W9yAmw7wSg== via 1.1 4a1f198d8af503c504dcbeb574c3a2a2.cloudfront.net (CloudFront)
GET H2	200	gtm.js Show response www.googletagmanager.com/	97 KB 32 KB	15ms 14ms	Script application/javascript	2a00:1450:4001:816::2008 Google Inc.
General Check archive.org Show headers Download Go to Full URL https://www.googletagmanager.com/gtm.js?id=GTM-KHWBBT Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 2a00:1450:4001:816::2008 , Ireland, ASN15169 (GOOGLE - Google Inc., US), Reverse DNS Software Google Tag Manager (scaffolding) / Resource Hash e79802ee4ad6c49e0bc49bc8cdd2b2ca881266b4d0188285931f25f64eb4796e Security Headers Name Value X-Xss-Protection 1; mode=block Request headers :path /gtm.js?id=GTM-KHWBBT pragma no-cache accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority www.googletagmanager.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:43 GMT content-encoding gzip server Google Tag Manager (scaffolding) access-control-allow-headers Cache-Control status 200 vary Accept-Encoding content-type application/javascript; charset=UTF-8 access-control-allow-origin http://www.googletagmanager.com cache-control private, max-age=900 access-control-allow-credentials true alt-svc quic=":443"; ma=2592000; v="39,38,37,35" content-length 32958 x-xss-protection 1; mode=block expires Mon, 23 Oct 2017 16:48:43 GMT
GET H2	200	icon-sprite-d76eb75d70ccf8ffb4c0b47b7dbc88ca.svg www.digitalocean.com/assets/community/	14 KB 4 KB	22ms 22ms	Other image/svg+xml	104.16.25.4 CloudFlare
General Check archive.org Show headers Download Go to Full URL https://www.digitalocean.com/assets/community/icon-sprite-d76eb75d70ccf8ffb4c0b47b7dbc88ca.svg Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 104.16.25.4 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash e6870c1bbc3155543c678190aeb5211e8d152e3597489365544ec6fdc9299257 Request headers :path /assets/community/icon-sprite-d76eb75d70ccf8ffb4c0b47b7dbc88ca.svg pragma no-cache cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; referrer=; _community_session=MGVndnNtNy9URy9TUW9vNm5CL0ZTQzBNajZuM2RRRHBMckFSVGpxcmd0TS9oSml0Z1RxQTNkbUdmeWV5YldRVDJIMENhbjdJZTRmaEkzUVRGR3hTdlFqaXlBZERJcGw1R2I0OXN6VnQycUNmMUxkZ2VBU3cvV3orTzQwck5hdFBTYTZtckJvYXB3VlB1MHZaVXNMNk01SjR2ckFoN3I0MjBHcWVxYW1LNHlheG1ITHhydmpyWXUwbGF2UXZuKzlNQXl1bUhNMEtLbEJ4a3FzcUV0cmJLR3NGZUdrOHBqWUpWZElnVVNIUVN2d3lyVzZDMTQ0R215aHBKdFpHYVRaUkdJVnc5QmRVVFozYTRReUErbmVTb2RXbWhXT3BrKzNMclY5MW5Td3Y0Uyt4bnIrVTRZbTZHelo1SE9jK1hxSVkwTzBlOGxBWXByUVIvSEM1eEthTmpBPT0tLTl3aGs1ZjNWUS8vTGhrTWVTRkVsSXc9PQ%3D%3D--a15aad9bbe5e226c0ee49dd0039533402d1085ae accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept image/webp,image/apng,image/*,*/*;q=0.8 cache-control no-cache :authority www.digitalocean.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:43 GMT content-encoding gzip cf-cache-status HIT last-modified Wed, 18 Jan 2017 18:57:32 GMT server cloudflare-nginx etag W/"587fba9c-372a" vary Accept-Encoding content-type image/svg+xml status 200 cache-control public, max-age=315360000 cf-ray 3b262f7f790f26ea-FRA expires Thu, 21 Oct 2027 16:48:43 GMT
GET H2	200	community_icons-ce0e079892b439e5bb4e9a66e68a92bf.woff www.digitalocean.com/assets/community/	20 KB 20 KB	22ms 22ms	Font application/octet-stream	104.16.25.4 CloudFlare
General Check archive.org Show headers Download Go to Full URL https://www.digitalocean.com/assets/community/community_icons-ce0e079892b439e5bb4e9a66e68a92bf.woff Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 104.16.25.4 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash e55152186db3c0202e1e0cd01a1a65801687197088d2a08fd2117bab94809659 Request headers :path /assets/community/community_icons-ce0e079892b439e5bb4e9a66e68a92bf.woff pragma no-cache cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; referrer=; _community_session=MGVndnNtNy9URy9TUW9vNm5CL0ZTQzBNajZuM2RRRHBMckFSVGpxcmd0TS9oSml0Z1RxQTNkbUdmeWV5YldRVDJIMENhbjdJZTRmaEkzUVRGR3hTdlFqaXlBZERJcGw1R2I0OXN6VnQycUNmMUxkZ2VBU3cvV3orTzQwck5hdFBTYTZtckJvYXB3VlB1MHZaVXNMNk01SjR2ckFoN3I0MjBHcWVxYW1LNHlheG1ITHhydmpyWXUwbGF2UXZuKzlNQXl1bUhNMEtLbEJ4a3FzcUV0cmJLR3NGZUdrOHBqWUpWZElnVVNIUVN2d3lyVzZDMTQ0R215aHBKdFpHYVRaUkdJVnc5QmRVVFozYTRReUErbmVTb2RXbWhXT3BrKzNMclY5MW5Td3Y0Uyt4bnIrVTRZbTZHelo1SE9jK1hxSVkwTzBlOGxBWXByUVIvSEM1eEthTmpBPT0tLTl3aGs1ZjNWUS8vTGhrTWVTRkVsSXc9PQ%3D%3D--a15aad9bbe5e226c0ee49dd0039533402d1085ae origin https://www.digitalocean.com accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority www.digitalocean.com referer https://www.digitalocean.com/assets/community/application-59796a7d20124872c204c9bc8c7193ef.css :scheme https :method GET User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Referer https://www.digitalocean.com/assets/community/application-59796a7d20124872c204c9bc8c7193ef.css Origin https://www.digitalocean.com Response headers date Mon, 23 Oct 2017 16:48:43 GMT cf-cache-status HIT last-modified Tue, 17 May 2016 17:55:14 GMT server cloudflare-nginx etag "573b5b02-50d8" vary Accept-Encoding content-type application/octet-stream status 200 cache-control public, max-age=315360000 accept-ranges bytes cf-ray 3b262f7f792826ea-FRA content-length 20696 expires Thu, 21 Oct 2027 16:48:43 GMT
POST H2	200	p Show response api.segment.io/v1/	21 B 39 B	500ms 166ms	XHR application/json	54.149.190.165 Amazon.com
General Check archive.org Show headers Download Go to Full URL https://api.segment.io/v1/p Requested by Host: cdn.segment.com URL: https://cdn.segment.com/analytics.js/v1/puo3uv968t/analytics.min.js Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 54.149.190.165 Boardman, United States, ASN16509 (AMAZON-02 - Amazon.com, Inc., US), Reverse DNS ec2-54-149-190-165.us-west-2.compute.amazonaws.com Software / Resource Hash 12f71cb993958eefc4bdb41d7dbbda490779a9c7aba448f7be52bb63912e0254 Request headers :path /v1/p pragma no-cache origin https://www.digitalocean.com accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 content-type text/plain accept */* cache-control no-cache :authority api.segment.io referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https content-length 1130 :method POST Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Origin https://www.digitalocean.com User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Content-Type text/plain Response headers status 200 date Mon, 23 Oct 2017 16:48:43 GMT access-control-allow-origin https://www.digitalocean.com content-length 21 vary Origin content-type application/json
GET H2	200	l use.typekit.net/af/610d30/00000000000000003b9ad1bb/27/	18 KB 18 KB	20ms 6ms	Font application/font-woff2	92.123.94.15 AKAMAI-ASN1
General Check archive.org Show headers Download Go to Full URL https://use.typekit.net/af/610d30/00000000000000003b9ad1bb/27/l?subset_id=2&fvd=n6&v=3 Requested by Host: use.typekit.net URL: https://use.typekit.net/izu1uqu.js Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_256_GCM Server 92.123.94.15 , European Union, ASN20940 (AKAMAI-ASN1, US), Reverse DNS a92-123-94-15.deploy.akamaitechnologies.com Software nginx / Resource Hash 4da8206845b9e15e5d86ce7e661c5c18666ce56c2377131aaec2a612e58804a5 Request headers :path /af/610d30/00000000000000003b9ad1bb/27/l?subset_id=2&fvd=n6&v=3 pragma no-cache origin https://www.digitalocean.com accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority use.typekit.net referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Origin https://www.digitalocean.com Response headers date Mon, 23 Oct 2017 16:48:43 GMT server nginx etag "80987524f2c82c2a36d727971941de8401d3f316" status 200 200 OK content-type application/font-woff2 access-control-allow-origin * cache-control public, max-age=8640000 timing-allow-origin * content-length 18688
GET H2	200	l use.typekit.net/af/1f8eae/00000000000000003b9ad1b9/27/	18 KB 18 KB	25ms 12ms	Font application/font-woff2	92.123.94.15 AKAMAI-ASN1
General Check archive.org Show headers Download Go to Full URL https://use.typekit.net/af/1f8eae/00000000000000003b9ad1b9/27/l?subset_id=2&fvd=n4&v=3 Requested by Host: use.typekit.net URL: https://use.typekit.net/izu1uqu.js Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_256_GCM Server 92.123.94.15 , European Union, ASN20940 (AKAMAI-ASN1, US), Reverse DNS a92-123-94-15.deploy.akamaitechnologies.com Software nginx / Resource Hash 22a314e594c21b9ad2d42fe9f2f5218d96d663d4d708ad89b0aa9efb5fac730a Request headers :path /af/1f8eae/00000000000000003b9ad1b9/27/l?subset_id=2&fvd=n4&v=3 pragma no-cache origin https://www.digitalocean.com accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority use.typekit.net referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Origin https://www.digitalocean.com Response headers date Mon, 23 Oct 2017 16:48:43 GMT server nginx etag "f9e85be3f0c8dcdcbd6f0a8471a46280ab7bf664" status 200 200 OK content-type application/font-woff2 access-control-allow-origin * cache-control public, max-age=8640000 timing-allow-origin * content-length 18496
GET H2	200	l use.typekit.net/af/ca838d/00000000000000003b9ad1ba/27/	19 KB 19 KB	24ms 11ms	Font application/font-woff2	92.123.94.15 AKAMAI-ASN1
General Check archive.org Show headers Download Go to Full URL https://use.typekit.net/af/ca838d/00000000000000003b9ad1ba/27/l?subset_id=2&fvd=i4&v=3 Requested by Host: use.typekit.net URL: https://use.typekit.net/izu1uqu.js Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_256_GCM Server 92.123.94.15 , European Union, ASN20940 (AKAMAI-ASN1, US), Reverse DNS a92-123-94-15.deploy.akamaitechnologies.com Software nginx / Resource Hash 4041f04f35d9b82a27d87141ef0f6b2c8c8f858ed51f4fa0170f266aa003a8fc Request headers :path /af/ca838d/00000000000000003b9ad1ba/27/l?subset_id=2&fvd=i4&v=3 pragma no-cache origin https://www.digitalocean.com accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority use.typekit.net referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Origin https://www.digitalocean.com Response headers date Mon, 23 Oct 2017 16:48:43 GMT server nginx etag "8887aa07a5e31ddeba60d1317cef52532c1e4862" status 200 200 OK content-type application/font-woff2 access-control-allow-origin * cache-control public, max-age=8640000 timing-allow-origin * content-length 19188
GET H2	200	l use.typekit.net/af/c2fb26/00000000000000003b9ad1b5/27/	18 KB 18 KB	25ms 11ms	Font application/font-woff2	92.123.94.15 AKAMAI-ASN1
General Check archive.org Show headers Download Go to Full URL https://use.typekit.net/af/c2fb26/00000000000000003b9ad1b5/27/l?subset_id=2&fvd=n3&v=3 Requested by Host: use.typekit.net URL: https://use.typekit.net/izu1uqu.js Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_256_GCM Server 92.123.94.15 , European Union, ASN20940 (AKAMAI-ASN1, US), Reverse DNS a92-123-94-15.deploy.akamaitechnologies.com Software nginx / Resource Hash 1d8d5156122647b1efe2df3b945e7674621f8f8cc9ee5ea2bbe1f24cc8c1c5c3 Request headers :path /af/c2fb26/00000000000000003b9ad1b5/27/l?subset_id=2&fvd=n3&v=3 pragma no-cache origin https://www.digitalocean.com accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority use.typekit.net referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Origin https://www.digitalocean.com Response headers date Mon, 23 Oct 2017 16:48:43 GMT server nginx etag "53497a4c5bfe1988b36f82f4d92f806e8f60ed2a" status 200 200 OK content-type application/font-woff2 access-control-allow-origin * cache-control public, max-age=8640000 timing-allow-origin * content-length 18460
GET H2	200	star-b061be59c343059d9802ecee86cfedc1.svg www.digitalocean.com/assets/community/icons/	142 B 156 B	21ms 21ms	Image image/svg+xml	104.16.25.4 CloudFlare
General Check archive.org Show headers Download Go to Show image Full URL https://www.digitalocean.com/assets/community/icons/star-b061be59c343059d9802ecee86cfedc1.svg Requested by Host: go.digitalocean.com URL: https://go.digitalocean.com/js/forms2/js/forms2.min.js Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 104.16.25.4 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash 683d807a7a8007af3a67cb3f1f7682cbb1bb5ed299e2dcc2f9293c2fc93f6180 Request headers :path /assets/community/icons/star-b061be59c343059d9802ecee86cfedc1.svg pragma no-cache cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; referrer=; _community_session=MGVndnNtNy9URy9TUW9vNm5CL0ZTQzBNajZuM2RRRHBMckFSVGpxcmd0TS9oSml0Z1RxQTNkbUdmeWV5YldRVDJIMENhbjdJZTRmaEkzUVRGR3hTdlFqaXlBZERJcGw1R2I0OXN6VnQycUNmMUxkZ2VBU3cvV3orTzQwck5hdFBTYTZtckJvYXB3VlB1MHZaVXNMNk01SjR2ckFoN3I0MjBHcWVxYW1LNHlheG1ITHhydmpyWXUwbGF2UXZuKzlNQXl1bUhNMEtLbEJ4a3FzcUV0cmJLR3NGZUdrOHBqWUpWZElnVVNIUVN2d3lyVzZDMTQ0R215aHBKdFpHYVRaUkdJVnc5QmRVVFozYTRReUErbmVTb2RXbWhXT3BrKzNMclY5MW5Td3Y0Uyt4bnIrVTRZbTZHelo1SE9jK1hxSVkwTzBlOGxBWXByUVIvSEM1eEthTmpBPT0tLTl3aGs1ZjNWUS8vTGhrTWVTRkVsSXc9PQ%3D%3D--a15aad9bbe5e226c0ee49dd0039533402d1085ae; ajs_user_id=null; ajs_group_id=null; ajs_anonymous_id=%22f7e35b82-9e38-47f0-bff5-319974a84824%22 accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept image/webp,image/apng,image/*,*/*;q=0.8 cache-control no-cache :authority www.digitalocean.com referer https://www.digitalocean.com/assets/community/application-59796a7d20124872c204c9bc8c7193ef.css :scheme https :method GET Referer https://www.digitalocean.com/assets/community/application-59796a7d20124872c204c9bc8c7193ef.css User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:43 GMT content-encoding gzip cf-cache-status HIT last-modified Tue, 31 May 2016 21:41:50 GMT server cloudflare-nginx etag W/"574e051e-8e" vary Accept-Encoding content-type image/svg+xml status 200 cache-control public, max-age=315360000 cf-ray 3b262f8019ce26ea-FRA expires Thu, 21 Oct 2027 16:48:43 GMT
GET H2	200	server_city-9f94d4e260564ce00c3d09e327a781f3.png www.digitalocean.com/assets/community/	30 KB 30 KB	24ms 24ms	Image image/png	104.16.25.4 CloudFlare
General Check archive.org Show headers Download Go to Show image Full URL https://www.digitalocean.com/assets/community/server_city-9f94d4e260564ce00c3d09e327a781f3.png Requested by Host: go.digitalocean.com URL: https://go.digitalocean.com/js/forms2/js/forms2.min.js Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 104.16.25.4 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash 1c7d311b620e0077427974d8c3d0be1c1b96f415ca40da18b1c5bba257c949be Request headers :path /assets/community/server_city-9f94d4e260564ce00c3d09e327a781f3.png pragma no-cache cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; referrer=; _community_session=MGVndnNtNy9URy9TUW9vNm5CL0ZTQzBNajZuM2RRRHBMckFSVGpxcmd0TS9oSml0Z1RxQTNkbUdmeWV5YldRVDJIMENhbjdJZTRmaEkzUVRGR3hTdlFqaXlBZERJcGw1R2I0OXN6VnQycUNmMUxkZ2VBU3cvV3orTzQwck5hdFBTYTZtckJvYXB3VlB1MHZaVXNMNk01SjR2ckFoN3I0MjBHcWVxYW1LNHlheG1ITHhydmpyWXUwbGF2UXZuKzlNQXl1bUhNMEtLbEJ4a3FzcUV0cmJLR3NGZUdrOHBqWUpWZElnVVNIUVN2d3lyVzZDMTQ0R215aHBKdFpHYVRaUkdJVnc5QmRVVFozYTRReUErbmVTb2RXbWhXT3BrKzNMclY5MW5Td3Y0Uyt4bnIrVTRZbTZHelo1SE9jK1hxSVkwTzBlOGxBWXByUVIvSEM1eEthTmpBPT0tLTl3aGs1ZjNWUS8vTGhrTWVTRkVsSXc9PQ%3D%3D--a15aad9bbe5e226c0ee49dd0039533402d1085ae; ajs_user_id=null; ajs_group_id=null; ajs_anonymous_id=%22f7e35b82-9e38-47f0-bff5-319974a84824%22 accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept image/webp,image/apng,image/*,*/*;q=0.8 cache-control no-cache :authority www.digitalocean.com referer https://www.digitalocean.com/assets/community/application-59796a7d20124872c204c9bc8c7193ef.css :scheme https :method GET Referer https://www.digitalocean.com/assets/community/application-59796a7d20124872c204c9bc8c7193ef.css User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:43 GMT cf-cache-status HIT last-modified Wed, 13 Jul 2016 12:59:53 GMT server cloudflare-nginx etag "57863b49-769e" vary Accept-Encoding content-type image/png status 200 cache-control public, max-age=315360000 accept-ranges bytes cf-ray 3b262f8019d026ea-FRA content-length 30366 expires Thu, 21 Oct 2027 16:48:43 GMT
GET H2	200	comments Show response www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3/	18 KB 3 KB	680ms 679ms	XHR text/javascript	104.16.25.4 CloudFlare
General Check archive.org Show headers Download Go to Full URL https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3/comments Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/assets/community/application-ed83ed45704e8c3619c0698b180fdc77.js Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 104.16.25.4 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash bc2d5623e9a51b21223c1f2a6ecdab669bf52ceb36e00f77e8683ea9900445a5 Security Headers Name Value X-Content-Type-Options nosniff X-Frame-Options SAMEORIGIN X-Xss-Protection 1; mode=block Request headers :path /community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3/comments pragma no-cache cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; referrer=; _community_session=MGVndnNtNy9URy9TUW9vNm5CL0ZTQzBNajZuM2RRRHBMckFSVGpxcmd0TS9oSml0Z1RxQTNkbUdmeWV5YldRVDJIMENhbjdJZTRmaEkzUVRGR3hTdlFqaXlBZERJcGw1R2I0OXN6VnQycUNmMUxkZ2VBU3cvV3orTzQwck5hdFBTYTZtckJvYXB3VlB1MHZaVXNMNk01SjR2ckFoN3I0MjBHcWVxYW1LNHlheG1ITHhydmpyWXUwbGF2UXZuKzlNQXl1bUhNMEtLbEJ4a3FzcUV0cmJLR3NGZUdrOHBqWUpWZElnVVNIUVN2d3lyVzZDMTQ0R215aHBKdFpHYVRaUkdJVnc5QmRVVFozYTRReUErbmVTb2RXbWhXT3BrKzNMclY5MW5Td3Y0Uyt4bnIrVTRZbTZHelo1SE9jK1hxSVkwTzBlOGxBWXByUVIvSEM1eEthTmpBPT0tLTl3aGs1ZjNWUS8vTGhrTWVTRkVsSXc9PQ%3D%3D--a15aad9bbe5e226c0ee49dd0039533402d1085ae; ajs_user_id=null; ajs_group_id=null; ajs_anonymous_id=%22f7e35b82-9e38-47f0-bff5-319974a84824%22 accept-encoding gzip, deflate x-csrf-token 9MioPKnn9Fr8nj+Ly5PV8Aoz9vO9Mfg5mOfuQxr6OfkvlSxIr4tkJyW1eR2B/HVRtDsOhw3lD16uXKU7MMB/hg== user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */*;q=0.5, text/javascript, application/javascript, application/ecmascript, application/x-ecmascript cache-control no-cache :authority www.digitalocean.com x-requested-with XMLHttpRequest :scheme https referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :method GET Accept */*;q=0.5, text/javascript, application/javascript, application/ecmascript, application/x-ecmascript Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 X-CSRF-Token 9MioPKnn9Fr8nj+Ly5PV8Aoz9vO9Mfg5mOfuQxr6OfkvlSxIr4tkJyW1eR2B/HVRtDsOhw3lD16uXKU7MMB/hg== User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 X-Requested-With XMLHttpRequest Response headers x-runtime 0.271451 date Mon, 23 Oct 2017 16:48:44 GMT content-encoding gzip x-content-type-options nosniff server cloudflare-nginx x-frame-options SAMEORIGIN content-type text/javascript; charset=utf-8 status 200 cache-control max-age=0, private, must-revalidate set-cookie _community_session=bHp1OE4wMENlL0NzTWtFSm82aUp3WEN4N3FienNzRGVKY08rdmlhbFdiQ1c0K2VsWEJSQkRsNkYzU2FOd2ovNWZTMjY5NG81UUZaRUJYYkljejFEOGVLWFhaREpPQmlxMlR2WUZ1MXg1UzlrbVA5dFV4VXhwclRkM2JVY2xLTk9pdGZiN096RTJhYktVUmlpWWk5a08wbGVEOGxMNGZQMG5lNjVPVGNGRXdBeDdrWm4zdXBtSWFSN21xZDJjL3FZa2F0MTFxSEw1enNVTVNLaXBJbVNHY0ZmUEw1dC90dDBoaGN0RXNnNStvdTEra1FJZlpnN2N4VW82YSs1ZWpxdlNNbHkwNG5Sc2F6NEtteGtEck5mZGYyUHN6TStoRDI3UlFjU3dEOUUxKzJGbVBmV0VZb1Z0a0xicUFhNmdFMnBrR202bjBnbG9CZWtGNjdIbUdLR1JRPT0tLTd2aHJNZ3lRQ2VFVDFmTWYvOFVsRkE9PQ%3D%3D--c772f52e56f0a8ba6f4762d7a9e525ae4688ebe4; path=/; HttpOnly cf-ray 3b262f806a6e26ea-FRA x-xss-protection 1; mode=block x-request-id dac0908e-8903-4b6a-8002-af76c408c8da
GET H2	200	icons-2cc887b5b0f833d3dc541895d7f6ba04.png www.digitalocean.com/assets/community/meltdown/	2 KB 2 KB	15ms 15ms	Image image/png	104.16.25.4 CloudFlare
General Check archive.org Show headers Download Go to Show image Full URL https://www.digitalocean.com/assets/community/meltdown/icons-2cc887b5b0f833d3dc541895d7f6ba04.png Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/assets/community/application-ed83ed45704e8c3619c0698b180fdc77.js Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 104.16.25.4 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash 799e67753782cf5aeb857d93c4e2727b2076d2ffa55ec68d23d011408a99dc50 Request headers :path /assets/community/meltdown/icons-2cc887b5b0f833d3dc541895d7f6ba04.png pragma no-cache cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; referrer=; _community_session=MGVndnNtNy9URy9TUW9vNm5CL0ZTQzBNajZuM2RRRHBMckFSVGpxcmd0TS9oSml0Z1RxQTNkbUdmeWV5YldRVDJIMENhbjdJZTRmaEkzUVRGR3hTdlFqaXlBZERJcGw1R2I0OXN6VnQycUNmMUxkZ2VBU3cvV3orTzQwck5hdFBTYTZtckJvYXB3VlB1MHZaVXNMNk01SjR2ckFoN3I0MjBHcWVxYW1LNHlheG1ITHhydmpyWXUwbGF2UXZuKzlNQXl1bUhNMEtLbEJ4a3FzcUV0cmJLR3NGZUdrOHBqWUpWZElnVVNIUVN2d3lyVzZDMTQ0R215aHBKdFpHYVRaUkdJVnc5QmRVVFozYTRReUErbmVTb2RXbWhXT3BrKzNMclY5MW5Td3Y0Uyt4bnIrVTRZbTZHelo1SE9jK1hxSVkwTzBlOGxBWXByUVIvSEM1eEthTmpBPT0tLTl3aGs1ZjNWUS8vTGhrTWVTRkVsSXc9PQ%3D%3D--a15aad9bbe5e226c0ee49dd0039533402d1085ae; ajs_user_id=null; ajs_group_id=null; ajs_anonymous_id=%22f7e35b82-9e38-47f0-bff5-319974a84824%22 accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept image/webp,image/apng,image/*,*/*;q=0.8 cache-control no-cache :authority www.digitalocean.com referer https://www.digitalocean.com/assets/community/application-59796a7d20124872c204c9bc8c7193ef.css :scheme https :method GET Referer https://www.digitalocean.com/assets/community/application-59796a7d20124872c204c9bc8c7193ef.css User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:43 GMT cf-cache-status HIT last-modified Mon, 23 May 2016 14:48:37 GMT server cloudflare-nginx etag "57431845-94c" vary Accept-Encoding content-type image/png status 200 cache-control public, max-age=315360000 accept-ranges bytes cf-ray 3b262f807a8e26ea-FRA content-length 2380 expires Thu, 21 Oct 2027 16:48:43 GMT
GET H/1.1	200 OK	widgets.js Show response platform.twitter.com/	121 KB 35 KB	6ms 6ms	Script application/javascript	2606:2800:234:59:254c:406:2366:268c MCI Communication...
General Check archive.org Show headers Download Go to Full URL https://platform.twitter.com/widgets.js Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/assets/community/application-ed83ed45704e8c3619c0698b180fdc77.js Protocol HTTP/1.1 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 2606:2800:234:59:254c:406:2366:268c , United States, ASN15133 (EDGECAST - MCI Communications Services, Inc. d/b/a Verizon Business, US), Reverse DNS Software ECS (fcn/41DE) / Resource Hash a111dafaebf131d73c8406a77a29d0b11438b759ebedf65360207555a2c3d854 Request headers Pragma no-cache Accept-Encoding gzip, deflate Host platform.twitter.com User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Accept */* Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Connection keep-alive Cache-Control no-cache Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers Date Mon, 23 Oct 2017 16:48:43 GMT Content-Encoding gzip Last-Modified Wed, 18 Oct 2017 15:59:14 GMT Server ECS (fcn/41DE) Etag "7206b71b83306cb84687a315b1a844eb+gzip" Vary Accept-Encoding X-Cache HIT P3P CP="CAO DSP LAW CURa ADMa DEVa TAIa PSAa PSDa IVAa IVDa OUR BUS IND UNI COM NAV INT" Cache-Control public, max-age=1800 Content-Type application/javascript; charset=utf-8 Content-Length 35450
GET H2	200	all.js Show response connect.facebook.net/en_US/	195 KB 61 KB	7ms 7ms	Script application/x-javascript	2a03:2880:f02d:12:face:b00c:0:3 Facebook
General Check archive.org Show headers Download Go to Full URL https://connect.facebook.net/en_US/all.js Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/assets/community/application-ed83ed45704e8c3619c0698b180fdc77.js Protocol H2 Security TLS 1.2, ECDHE_ECDSA, AES_128_GCM Server 2a03:2880:f02d:12:face:b00c:0:3 , Ireland, ASN32934 (FACEBOOK - Facebook, Inc., US), Reverse DNS Software / Resource Hash b0dc51bf310905b73b49ac3c777b8bb292c36cba4efbd07e3891f3a368b9de8b Security Headers Name Value Content-Security-Policy default-src * data: blob:;script-src *.facebook.com *.fbcdn.net *.facebook.net *.google-analytics.com *.virtualearth.net *.google.com 127.0.0.1:* *.spotilocal.com:* 'unsafe-inline' 'unsafe-eval' fbstatic-a.akamaihd.net fbcdn-static-b-a.akamaihd.net *.atlassolutions.com blob: data: 'self';style-src data: blob: 'unsafe-inline' *;connect-src *.facebook.com *.fbcdn.net *.facebook.net *.spotilocal.com:* *.akamaihd.net wss://*.facebook.com:* https://fb.scanandcleanlocal.com:* *.atlassolutions.com attachment.fbsbx.com ws://localhost:* blob: *.cdninstagram.com 'self' chrome-extension://boadgeojelhgndaghljhdicfkmllpafd chrome-extension://dliochdbjfkdbacpmhlcpmleaejidimm; Strict-Transport-Security max-age=15552000; preload; includeSubDomains X-Content-Type-Options nosniff X-Frame-Options DENY X-Xss-Protection 0 Request headers :path /en_US/all.js pragma no-cache accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority connect.facebook.net referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers strict-transport-security max-age=15552000; preload; includeSubDomains content-encoding gzip x-content-type-options nosniff content-md5 /JO32NyYKzfaOLp/FjizGw== status 200 content-length 62264 x-xss-protection 0 x-fb-debug G0SUA7p0RKzc/6xtFYvfHsowVVkmfssjLu83maRPRA0Q+AdXPomY3Rsei/vZccv0xmB0GN18BO1kBl3ffIsGMg== x-fb-content-md5 14ebe01e45e71ae24b1d0b14636bff8c x-frame-options DENY date Mon, 23 Oct 2017 16:48:43 GMT expect-ct max-age=10, report-uri="http://reports.fb.com/expectct/" vary Accept-Encoding content-type application/x-javascript; charset=utf-8 access-control-expose-headers X-FB-Content-MD5 cache-control public,max-age=1200,stale-while-revalidate=3600 etag "bbfa4fb380f99c061261958b9f437e75" content-security-policy default-src * data: blob:;script-src *.facebook.com *.fbcdn.net *.facebook.net *.google-analytics.com *.virtualearth.net *.google.com 127.0.0.1:* *.spotilocal.com:* 'unsafe-inline' 'unsafe-eval' fbstatic-a.akamaihd.net fbcdn-static-b-a.akamaihd.net *.atlassolutions.com blob: data: 'self';style-src data: blob: 'unsafe-inline' *;connect-src *.facebook.com *.fbcdn.net *.facebook.net *.spotilocal.com:* *.akamaihd.net wss://*.facebook.com:* https://fb.scanandcleanlocal.com:* *.atlassolutions.com attachment.fbsbx.com ws://localhost:* blob: *.cdninstagram.com 'self' chrome-extension://boadgeojelhgndaghljhdicfkmllpafd chrome-extension://dliochdbjfkdbacpmhlcpmleaejidimm; timing-allow-origin * expires Mon, 23 Oct 2017 17:04:58 GMT
GET H2	200	getForm Show response go.digitalocean.com/index.php/form/	3 KB 1 KB	700ms 700ms	Script application/javascript	104.16.111.208 CloudFlare
General Check archive.org Show headers Download Go to Full URL https://go.digitalocean.com/index.php/form/getForm?munchkinId=937-EID-756&form=1071&url=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3&callback=jQuery110200687281313320014_1508777323322&_=1508777323323 Requested by Host: go.digitalocean.com URL: https://go.digitalocean.com/js/forms2/js/forms2.min.js Protocol H2 Security TLS 1.2, ECDHE_ECDSA, AES_128_GCM Server 104.16.111.208 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash e9a61133fc79da06aeaa8e8f3608d6be787edaeefa5801a86e3381cf576de247 Security Headers Name Value X-Content-Type-Options nosniff Request headers :path /index.php/form/getForm?munchkinId=937-EID-756&form=1071&url=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3&callback=jQuery110200687281313320014_1508777323322&_=1508777323323 pragma no-cache cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; referrer=; ajs_user_id=null; ajs_group_id=null; ajs_anonymous_id=%22f7e35b82-9e38-47f0-bff5-319974a84824%22 accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority go.digitalocean.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:44 GMT content-encoding gzip x-content-type-options nosniff server cloudflare-nginx content-type application/javascript; charset=utf-8 status 200 set-cookie BIGipServerab16web-app_https=!ZALf84qjAUFz2CkNEbaWaFcUiNHQQqPWchZXu7pTpp7OoAhnI62ZfCRWlBmExvDl0jRu+B6ooSBwZVY=; path=/; Httponly; Secure cf-ray 3b262f80a9c663a9-FRA
GET H2	200	qevents.js Show response a.quora.com/	23 KB 8 KB	24ms 7ms	Script text/plain	192.229.221.122 MCI Communication...
General Check archive.org Show headers Download Go to Full URL https://a.quora.com/qevents.js Requested by Host: cdn.segment.com URL: https://cdn.segment.com/analytics.js/v1/puo3uv968t/analytics.min.js Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 192.229.221.122 , United States, ASN15133 (EDGECAST - MCI Communications Services, Inc. d/b/a Verizon Business, US), Reverse DNS Software ECAcc (frc/8F77) / Resource Hash c6330783479f47565d40627db910e3f4f42283a302cb2377947d7db44e912a79 Request headers :path /qevents.js pragma no-cache accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority a.quora.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:43 GMT content-encoding gzip last-modified Thu, 21 Sep 2017 20:25:44 GMT server ECAcc (frc/8F77) x-amz-request-id E07E10F62774A0E9 etag "3e67e6e82b90756bbcb5249b6ba080b3+gzip" vary Accept-Encoding x-cache HIT content-type text/plain; charset=us-ascii status 200 x-amz-version-id 4X0phLQ2BQCVneZWKlLNDyFUzkgXIsq0 content-length 7689 x-amz-id-2 huNcSDj2Pg0R8cBpXTT9mNkaImBScoi50NUH8Aqm8pZOo6N7RsObO0gc1ofYjje9EkI0OQAQFAE=
GET H/1.1	200 OK	p.gif p.typekit.net/	35 B 35 B	21ms 6ms	Image image/gif	2a02:26f0:122:385::20c1 AKAMAI-ASN1
General Check archive.org Show headers Download Go to Show image Full URL https://p.typekit.net/p.gif?s=1&k=izu1uqu&ht=tk&h=www.digitalocean.com&f=173.175.176.5474&a=610424&js=1.18.24&app=typekit&e=js&_=1508777323691 Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol HTTP/1.1 Security TLS 1.2, ECDHE_RSA, AES_256_GCM Server 2a02:26f0:122:385::20c1 , European Union, ASN20940 (AKAMAI-ASN1, US), Reverse DNS Software nginx / Resource Hash 9b9265c69a5cc295d1ab0d04e0273b3677db1a6216ce2ccf4efc8c277ed84b39 Request headers Pragma no-cache Accept-Encoding gzip, deflate Host p.typekit.net User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Accept image/webp,image/apng,image/*,*/*;q=0.8 Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Connection keep-alive Cache-Control no-cache Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers Date Mon, 23 Oct 2017 16:48:43 GMT Last-Modified Thu, 17 Nov 2016 16:43:04 GMT Server nginx ETag "582dde18-23" Content-Type image/gif Access-Control-Allow-Origin * Cache-Control max-age=604800 Connection keep-alive Accept-Ranges bytes Content-Length 35 Expires Mon, 19 Jun 2017 06:55:06 GMT
GET		twitter_cookies.html platform.twitter.com/widgets/ Frame 2130	0 0
GET H2	200	settings Show response syndication.twitter.com/	57 B 91 B	110ms 109ms	Fetch application/json	104.244.42.200 Twitter Inc.
General Check archive.org Show headers Download Go to Full URL https://syndication.twitter.com/settings Requested by Host: platform.twitter.com URL: https://platform.twitter.com/widgets.js Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 104.244.42.200 San Francisco, United States, ASN13414 (TWITTER - Twitter Inc., US), Reverse DNS Software tsa_o / Resource Hash d442331ca710bdda5dfc13b7f65f78d601d0f9576d83a9eb1e628dcbbbbb2ef6 Security Headers Name Value Strict-Transport-Security max-age=631138519 Request headers :path /settings pragma no-cache origin https://www.digitalocean.com accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority syndication.twitter.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Origin https://www.digitalocean.com Response headers x-response-time 103 date Mon, 23 Oct 2017 16:48:43 GMT content-encoding gzip last-modified Mon, 23 Oct 2017 16:48:43 GMT server tsa_o status 200 vary Origin content-type application/json; charset=utf-8 access-control-allow-origin https://www.digitalocean.com cache-control must-revalidate, max-age=600 access-control-allow-credentials true x-connection-hash c6068e2ff6b676a73f6fecfc5f3c4fbf strict-transport-security max-age=631138519 content-length 82
GET H/1.1	200 OK	Cookie set pixel q.quora.com/_/ad/b38c184aa72c43ef8fb074e64602b64e/	43 B 43 B	421ms 101ms	Image image/gif	34.228.104.199 Amazon.com
General Check archive.org Show headers Download Go to Show image Full URL https://q.quora.com/_/ad/b38c184aa72c43ef8fb074e64602b64e/pixel?j=1&u=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3&tag=ViewContent&ts=1508777323760 Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol HTTP/1.1 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 34.228.104.199 Ashburn, United States, ASN14618 (AMAZON-AES - Amazon.com, Inc., US), Reverse DNS ec2-34-228-104-199.compute-1.amazonaws.com Software nginx / Resource Hash 548f2d6f4d0d820c6c5ffbeffcbd7f0e73193e2932eefe542accc84762deec87 Request headers Pragma no-cache Accept-Encoding gzip, deflate Host q.quora.com User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Accept image/webp,image/apng,image/*,*/*;q=0.8 Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Connection keep-alive Cache-Control no-cache Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers Set-Cookie m-b="RfNxRZTUco5huwWpNh-QjQ=="; expires=Thur, 09 Nov 2035 23:58:59 GMT; HttpOnly; Path=/; Secure Date Mon, 23 Oct 2017 16:48:44 GMT Server nginx Connection keep-alive Content-Length 43 Content-Type image/gif
GET H2	200	/ www.facebook.com/impression.php/f2c96a6e3b15d68/	43 B 66 B	123ms 123ms	Image image/gif	2a03:2880:f12d:83:face:b00c:0:25de Facebook
General Check archive.org Show headers Download Go to Show image Full URL https://www.facebook.com/impression.php/f2c96a6e3b15d68/?api_key=694818843983011&lid=115&payload=%7B%22source%22%3A%22jssdk%22%7D Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol H2 Security TLS 1.2, ECDHE_ECDSA, AES_128_GCM Server 2a03:2880:f12d:83:face:b00c:0:25de , Ireland, ASN32934 (FACEBOOK - Facebook, Inc., US), Reverse DNS Software / Resource Hash 548f2d6f4d0d820c6c5ffbeffcbd7f0e73193e2932eefe542accc84762deec87 Security Headers Name Value Content-Security-Policy default-src * data: blob:;script-src *.facebook.com *.fbcdn.net *.facebook.net *.google-analytics.com *.virtualearth.net *.google.com 127.0.0.1:* *.spotilocal.com:* 'unsafe-inline' 'unsafe-eval' fbstatic-a.akamaihd.net fbcdn-static-b-a.akamaihd.net *.atlassolutions.com blob: data: 'self';style-src data: blob: 'unsafe-inline' *;connect-src *.facebook.com *.fbcdn.net *.facebook.net *.spotilocal.com:* *.akamaihd.net wss://*.facebook.com:* https://fb.scanandcleanlocal.com:* *.atlassolutions.com attachment.fbsbx.com ws://localhost:* blob: *.cdninstagram.com 'self' chrome-extension://boadgeojelhgndaghljhdicfkmllpafd chrome-extension://dliochdbjfkdbacpmhlcpmleaejidimm; Strict-Transport-Security max-age=15552000; preload X-Content-Type-Options nosniff X-Xss-Protection 0 Request headers :path /impression.php/f2c96a6e3b15d68/?api_key=694818843983011&lid=115&payload=%7B%22source%22%3A%22jssdk%22%7D pragma no-cache accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept image/webp,image/apng,image/*,*/*;q=0.8 cache-control no-cache :authority www.facebook.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers content-security-policy default-src * data: blob:;script-src *.facebook.com *.fbcdn.net *.facebook.net *.google-analytics.com *.virtualearth.net *.google.com 127.0.0.1:* *.spotilocal.com:* 'unsafe-inline' 'unsafe-eval' fbstatic-a.akamaihd.net fbcdn-static-b-a.akamaihd.net *.atlassolutions.com blob: data: 'self';style-src data: blob: 'unsafe-inline' *;connect-src *.facebook.com *.fbcdn.net *.facebook.net *.spotilocal.com:* *.akamaihd.net wss://*.facebook.com:* https://fb.scanandcleanlocal.com:* *.atlassolutions.com attachment.fbsbx.com ws://localhost:* blob: *.cdninstagram.com 'self' chrome-extension://boadgeojelhgndaghljhdicfkmllpafd chrome-extension://dliochdbjfkdbacpmhlcpmleaejidimm; content-encoding gzip x-content-type-options nosniff status 200 vary Origin Accept-Encoding x-xss-protection 0 pragma no-cache x-fb-debug 9fDYlm9mhobOIZ4GABpJrESG/+ajnh9QJRb8SrHUQQ1CU3BGXx5fVXXMDcKMQ1yp3nQyDHWUaGAmjO227c2Z2w== date Mon, 23 Oct 2017 16:48:43 GMT expect-ct max-age=10, report-uri="http://reports.fb.com/expectct/" strict-transport-security max-age=15552000; preload public-key-pins-report-only max-age=600; pin-sha256="WoiWRyIOVNa9ihaBciRSC7XHjliYS9VwUGOIud4PB18="; pin-sha256="k2v657xBsOVe1PQRwOsHsw3bsGT2VzIqz5K+59sNQws="; pin-sha256="gMxWOrX4PMQesK9qFNbYBxjBfjUvlkn/vN1n+L9lE5E="; pin-sha256="q4PO2G2cbkZhZ82+JgmRUyGMoAeozA+BSXVXQWB8XWQ="; includeSubdomains; report-uri="http://reports.fb.com/hpkp/" access-control-allow-origin https://www.facebook.com access-control-expose-headers X-FB-Debug, X-Loader-Length cache-control private, no-cache, no-store, must-revalidate access-control-allow-credentials true content-type image/gif access-control-allow-method OPTIONS expires Sat, 01 Jan 2000 00:00:00 GMT
GET		hsBwMj6iLmk.js staticxx.facebook.com/connect/xd_arbiter/r/ Frame 2130	0 0
GET H/1.1	200 OK	Hacktoberfest17-CommBanner-03.png hacktoberfest.nyc3.digitaloceanspaces.com/	93 KB 93 KB	404ms 127ms	Image image/png	162.243.189.2
General Check archive.org Show headers Download Go to Show image Full URL https://hacktoberfest.nyc3.digitaloceanspaces.com/Hacktoberfest17-CommBanner-03.png Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/assets/community/application-ed83ed45704e8c3619c0698b180fdc77.js Protocol HTTP/1.1 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 162.243.189.2 New York, United States, ASN (), Reverse DNS Software / Resource Hash 86b9b74b8fd3427967a2d2540a3fa29774351c7984daaa88f7916c604f465bc5 Security Headers Name Value Strict-Transport-Security max-age=15552000; includeSubDomains; preload Request headers Pragma no-cache Accept-Encoding gzip, deflate Host hacktoberfest.nyc3.digitaloceanspaces.com User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Accept image/webp,image/apng,image/*,*/*;q=0.8 Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Connection keep-alive Cache-Control no-cache Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers Date Mon, 23 Oct 2017 16:48:44 GMT Last-Modified Thu, 28 Sep 2017 13:32:41 GMT x-amz-request-id tx000000000000003200095-0059ee1d6c-37447-nyc3a ETag "41768f534bbb1077d6c1a12280161333" Strict-Transport-Security max-age=15552000; includeSubDomains; preload Content-Type image/png Accept-Ranges bytes Content-Length 95662
GET H2	200	forms2.css go.digitalocean.com/js/forms2/css/	13 KB 3 KB	10ms 10ms	Stylesheet text/css	104.16.111.208 CloudFlare
General Check archive.org Show headers Download Go to Full URL https://go.digitalocean.com/js/forms2/css/forms2.css Requested by Host: go.digitalocean.com URL: https://go.digitalocean.com/js/forms2/js/forms2.min.js Protocol H2 Security TLS 1.2, ECDHE_ECDSA, AES_128_GCM Server 104.16.111.208 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash 397d07fbfb19b6ac538d7b8bcdf5ebf7be881c9f9ad3982278d9d4f3a02c160b Security Headers Name Value X-Content-Type-Options nosniff Request headers :path /js/forms2/css/forms2.css pragma no-cache cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; referrer=; ajs_user_id=null; ajs_group_id=null; ajs_anonymous_id=%22f7e35b82-9e38-47f0-bff5-319974a84824%22; BIGipServerab16web-app_https=!ZALf84qjAUFz2CkNEbaWaFcUiNHQQqPWchZXu7pTpp7OoAhnI62ZfCRWlBmExvDl0jRu+B6ooSBwZVY= accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept text/css,*/*;q=0.1 cache-control no-cache :authority go.digitalocean.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:44 GMT content-encoding gzip x-content-type-options nosniff cf-cache-status HIT last-modified Fri, 07 Apr 2017 19:34:58 GMT server cloudflare-nginx etag "32038e-33f8-54c98b884bc80" vary Accept-Encoding content-type text/css status 200 cache-control public, max-age=7200 accept-ranges bytes cf-ray 3b262f850f3663a9-FRA content-length 2610 expires Mon, 23 Oct 2017 18:48:44 GMT
GET H2	200	forms2-theme-simple.css go.digitalocean.com/js/forms2/css/	826 B 260 B	18ms 17ms	Stylesheet text/css	104.16.111.208 CloudFlare
General Check archive.org Show headers Download Go to Full URL https://go.digitalocean.com/js/forms2/css/forms2-theme-simple.css Requested by Host: go.digitalocean.com URL: https://go.digitalocean.com/js/forms2/js/forms2.min.js Protocol H2 Security TLS 1.2, ECDHE_ECDSA, AES_128_GCM Server 104.16.111.208 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash 14c8c62dc692fd8faa04434e3fed25e7c23d596b732f9db88f6e9f9ff5dfa61c Security Headers Name Value X-Content-Type-Options nosniff Request headers :path /js/forms2/css/forms2-theme-simple.css pragma no-cache cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; referrer=; ajs_user_id=null; ajs_group_id=null; ajs_anonymous_id=%22f7e35b82-9e38-47f0-bff5-319974a84824%22; BIGipServerab16web-app_https=!ZALf84qjAUFz2CkNEbaWaFcUiNHQQqPWchZXu7pTpp7OoAhnI62ZfCRWlBmExvDl0jRu+B6ooSBwZVY= accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept text/css,*/*;q=0.1 cache-control no-cache :authority go.digitalocean.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:44 GMT content-encoding gzip x-content-type-options nosniff cf-cache-status HIT last-modified Fri, 07 Apr 2017 19:34:58 GMT server cloudflare-nginx etag "32038f-33a-54c98b884bc80" vary Accept-Encoding content-type text/css status 200 cache-control public, max-age=7200 accept-ranges bytes cf-ray 3b262f850f3763a9-FRA content-length 242 expires Mon, 23 Oct 2017 18:48:44 GMT
GET H2	200	XDFrame Show response go.digitalocean.com/index.php/form/ Frame 2130	2 KB 653 B	625ms 624ms	Document text/html	104.16.111.208 CloudFlare
General Check archive.org Show headers Download Go to Full URL https://go.digitalocean.com/index.php/form/XDFrame Requested by Host: go.digitalocean.com URL: https://go.digitalocean.com/js/forms2/js/forms2.min.js Protocol H2 Security TLS 1.2, ECDHE_ECDSA, AES_128_GCM Server 104.16.111.208 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash 8d6a6a3efc14bfafb9b47776673ec1bdfe0d79616b2a755b42f89c1f474d2171 Security Headers Name Value X-Content-Type-Options nosniff Request headers :path /index.php/form/XDFrame pragma no-cache cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; referrer=; ajs_user_id=null; ajs_group_id=null; ajs_anonymous_id=%22f7e35b82-9e38-47f0-bff5-319974a84824%22; BIGipServerab16web-app_https=!ZALf84qjAUFz2CkNEbaWaFcUiNHQQqPWchZXu7pTpp7OoAhnI62ZfCRWlBmExvDl0jRu+B6ooSBwZVY= accept-encoding gzip, deflate upgrade-insecure-requests 1 user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8 cache-control no-cache :authority go.digitalocean.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Upgrade-Insecure-Requests 1 Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:44 GMT content-encoding gzip x-content-type-options nosniff server cloudflare-nginx vary Accept-Encoding content-type text/html; charset=utf-8 status 200 cf-ray 3b262f855f8663a9-FRA content-length 635
GET H2	200	forms2.min.js Show response go.digitalocean.com/js/forms2/js/ Frame 2130	165 KB 0	95ms 64ms	Script application/x-javascript	104.16.111.208 CloudFlare
General Check archive.org Show headers Download Go to Full URL https://go.digitalocean.com/js/forms2/js/forms2.min.js Requested by Host: go.digitalocean.com URL: https://go.digitalocean.com/index.php/form/XDFrame Protocol H2 Security TLS 1.2, ECDHE_ECDSA, AES_128_GCM Server 104.16.111.208 , United States, ASN13335 (CLOUDFLARENET - CloudFlare, Inc., US), Reverse DNS Software cloudflare-nginx / Resource Hash aaee78be73219813ee518842197fffc34bc09d755f52f4e829fd8ffec460f876 Security Headers Name Value X-Content-Type-Options nosniff Request headers :path /js/forms2/js/forms2.min.js pragma no-cache cookie __cfduid=dc25d7811680327e13bdb33042fb930b01508777320; first_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; last_landing_page=%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3; referrer= accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority go.digitalocean.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Response headers date Mon, 23 Oct 2017 16:48:43 GMT content-encoding gzip x-content-type-options nosniff cf-cache-status HIT last-modified Mon, 25 Sep 2017 19:07:26 GMT server cloudflare-nginx etag "32039b-292eb-55a0844ea5780" vary Accept-Encoding content-type application/x-javascript status 200 cache-control public, max-age=7200 cf-ray 3b262f7e3e5b63a9-FRA expires Mon, 23 Oct 2017 18:48:43 GMT
POST H2	200	t Show response api.segment.io/v1/	21 B 39 B	166ms 165ms	XHR application/json	54.149.190.165 Amazon.com
General Check archive.org Show headers Download Go to Full URL https://api.segment.io/v1/t Requested by Host: cdn.segment.com URL: https://cdn.segment.com/analytics.js/v1/puo3uv968t/analytics.min.js Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 54.149.190.165 Boardman, United States, ASN16509 (AMAZON-02 - Amazon.com, Inc., US), Reverse DNS ec2-54-149-190-165.us-west-2.compute.amazonaws.com Software / Resource Hash 12f71cb993958eefc4bdb41d7dbbda490779a9c7aba448f7be52bb63912e0254 Request headers :path /v1/t pragma no-cache origin https://www.digitalocean.com accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 content-type text/plain accept */* cache-control no-cache :authority api.segment.io referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https content-length 1274 :method POST Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Origin https://www.digitalocean.com User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Content-Type text/plain Response headers status 200 date Mon, 23 Oct 2017 16:48:45 GMT access-control-allow-origin https://www.digitalocean.com content-length 21 vary Origin content-type application/json
GET		hsBwMj6iLmk.js staticxx.facebook.com/connect/xd_arbiter/r/ Frame 2130 Redirect Chain https://www.facebook.com/connect/ping?client_id=694818843983011&domain=www.digitalocean.com&origin=1&redirect_uri=https%3A%2F%2Fstaticxx.facebook.com%2Fconnect%2Fxd_arbiter%2Fr%2FhsBwMj6iLmk.js%3Fv... https://staticxx.facebook.com/connect/xd_arbiter/r/hsBwMj6iLmk.js?version=42	0 0
GET H2	200	analytics.js Show response www.google-analytics.com/	34 KB 14 KB	6ms 5ms	Script text/javascript	2a00:1450:4001:816::200e Google Inc.
General Check archive.org Show headers Download Go to Full URL https://www.google-analytics.com/analytics.js Requested by Host: www.googletagmanager.com URL: https://www.googletagmanager.com/gtm.js?id=GTM-KHWBBT Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 2a00:1450:4001:816::200e , Ireland, ASN15169 (GOOGLE - Google Inc., US), Reverse DNS Software Golfe2 / Resource Hash c6b51278f1a5a919cbc532ab29d06e1b1a918ee779cd055d27fc07120fd9093e Security Headers Name Value Strict-Transport-Security max-age=10886400; includeSubDomains; preload X-Content-Type-Options nosniff Request headers :path /analytics.js pragma no-cache accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority www.google-analytics.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers strict-transport-security max-age=10886400; includeSubDomains; preload content-encoding gzip x-content-type-options nosniff last-modified Thu, 28 Sep 2017 22:31:34 GMT server Golfe2 age 6338 date Mon, 23 Oct 2017 15:03:07 GMT vary Accept-Encoding content-type text/javascript status 200 cache-control public, max-age=7200 timing-allow-origin * alt-svc quic=":443"; ma=2592000; v="39,38,37,35" content-length 14089 expires Mon, 23 Oct 2017 17:03:07 GMT
GET H/1.1	200 OK	014ab3bd.min.js Show response scripts.demandbase.com/	56 KB 14 KB	203ms 14ms	Script application/javascript	52.85.254.74 Amazon.com
General Check archive.org Show headers Download Go to Full URL https://scripts.demandbase.com/014ab3bd.min.js Requested by Host: www.digitalocean.com URL: https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol HTTP/1.1 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 52.85.254.74 Seattle, United States, ASN16509 (AMAZON-02 - Amazon.com, Inc., US), Reverse DNS server-52-85-254-74.ams1.r.cloudfront.net Software AmazonS3 / Resource Hash 90cb3fba65dc882c39dc90cb6557bd571af9822b9c411228bd894835b691770f Request headers Pragma no-cache Accept-Encoding gzip, deflate Host scripts.demandbase.com User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Accept */* Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Connection keep-alive Cache-Control no-cache Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers Date Thu, 12 Oct 2017 17:35:31 GMT Content-Encoding gzip Last-Modified Thu, 12 Oct 2017 00:49:08 GMT Server AmazonS3 Age 658 Vary Accept-Encoding X-Cache Hit from cloudfront x-amz-version-id 8Cr9Z76QXa4vAYWAwBYWLH0ykfxcI3d9 Via 1.1 ac987789ab8e4a7dbf75086d523e8589.cloudfront.net (CloudFront) Cache-Control public, max-age=3600 Transfer-Encoding chunked Connection keep-alive Content-Type application/javascript X-Amz-Cf-Id aWfC7OY1SJZThhURGoT6vwk-BXzee024Wo0PVL7Eo8wJV2TebQ_o8g==
GET H2	200	tr www.facebook.com/	44 B 53 B	7ms 6ms	Image image/gif	2a03:2880:f12d:83:face:b00c:0:25de Facebook
General Check archive.org Show headers Download Go to Show image Full URL https://www.facebook.com/tr?id=1428881624071898&ev=PageView&noscript=1&gtmcb=1226952190 Protocol H2 Security TLS 1.2, ECDHE_ECDSA, AES_128_GCM Server 2a03:2880:f12d:83:face:b00c:0:25de , Ireland, ASN32934 (FACEBOOK - Facebook, Inc., US), Reverse DNS Software proxygen / Resource Hash 10d8d42d73a02ddb877101e72fbfa15a0ec820224d97cedee4cf92d571be5caa Request headers :path /tr?id=1428881624071898&ev=PageView&noscript=1&gtmcb=1226952190 pragma no-cache accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept image/webp,image/apng,image/*,*/*;q=0.8 cache-control no-cache :authority www.facebook.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:45 GMT last-modified Fri, 21 Dec 2012 00:00:01 GMT server proxygen content-type image/gif status 200 cache-control no-cache, must-revalidate, max-age=0 set-cookie fr=0rWtTag8M4TmNG8B8..BZ7h1t...1.0.BZ7h1t.; expires=Sunday, 21-Jan-2018 16:48:45 GMT; path=/; domain=.facebook.com; HttpOnly; secure content-length 44 expires Mon, 23 Oct 2017 16:48:45 GMT
GET H2	200	adsct analytics.twitter.com/i/	43 B 74 B	128ms 111ms	Image image/gif	104.244.42.67 Twitter Inc.
General Check archive.org Show headers Download Go to Show image Full URL https://analytics.twitter.com/i/adsct?txn_id=nuqqc&p_id=Twitter&tw_sale_amount=0&tw_order_quantity=0 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 104.244.42.67 San Francisco, United States, ASN13414 (TWITTER - Twitter Inc., US), Reverse DNS Software tsa_o / Resource Hash ac8778041fdb7f2e08ceb574c9a766247ea26f1a7d90fa854c4efcf4b361a957 Security Headers Name Value Strict-Transport-Security max-age=631138519 X-Content-Type-Options nosniff X-Frame-Options SAMEORIGIN X-Xss-Protection 1; mode=block Request headers :path /i/adsct?txn_id=nuqqc&p_id=Twitter&tw_sale_amount=0&tw_order_quantity=0 pragma no-cache accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept image/webp,image/apng,image/*,*/*;q=0.8 cache-control no-cache :authority analytics.twitter.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:45 GMT content-encoding gzip x-content-type-options nosniff status 200 200 OK x-twitter-response-tags BouncerCompliant x-connection-hash 4359c73d47ebccbcf9ecae7dc1756817 content-length 65 x-xss-protection 1; mode=block x-response-time 105 pragma no-cache last-modified Mon, 23 Oct 2017 16:48:45 GMT server tsa_o x-frame-options SAMEORIGIN strict-transport-security max-age=631138519 content-type image/gif;charset=utf-8 cache-control no-cache, no-store, must-revalidate, pre-check=0, post-check=0 set-cookie personalization_id="v1_udyd2Q9+G7hoWuGygGfzVQ=="; Expires=Wed, 23 Oct 2019 16:48:45 UTC; Path=/; Domain=.twitter.com guest_id=v1%3A150877732512667933; Expires=Wed, 23 Oct 2019 16:48:45 UTC; Path=/; Domain=.twitter.com x-transaction 0052cf2f002b23aa expires Tue, 31 Mar 1981 05:00:00 GMT
GET H2	200	linkid.js Show response www.google-analytics.com/plugins/ua/	2 KB 865 B	6ms 5ms	Script text/javascript	2a00:1450:4001:816::200e Google Inc.
General Check archive.org Show headers Download Go to Full URL https://www.google-analytics.com/plugins/ua/linkid.js Requested by Host: www.google-analytics.com URL: https://www.google-analytics.com/analytics.js Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 2a00:1450:4001:816::200e , Ireland, ASN15169 (GOOGLE - Google Inc., US), Reverse DNS Software sffe / Resource Hash 92fca55833f48b4289ac8f1cedd48752b580fce4ec4b5d81670b8193d6e51b54 Security Headers Name Value X-Content-Type-Options nosniff X-Xss-Protection 1; mode=block Request headers :path /plugins/ua/linkid.js pragma no-cache accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept */* cache-control no-cache :authority www.google-analytics.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 15:58:52 GMT content-encoding gzip x-content-type-options nosniff last-modified Thu, 21 Apr 2016 03:17:22 GMT server sffe age 2993 vary Accept-Encoding content-type text/javascript status 200 cache-control public, max-age=3600 accept-ranges bytes alt-svc quic=":443"; ma=2592000; v="39,38,37,35" content-length 856 x-xss-protection 1; mode=block expires Mon, 23 Oct 2017 16:58:52 GMT
GET H2	200	ga-audiences www.google.de/ads/ Redirect Chain https://www.google-analytics.com/r/collect?v=1&_v=j64&a=174397586&t=pageview&_s=1&dl=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3&ul... https://stats.g.doubleclick.net/r/collect?v=1&aip=1&t=dc&_r=3&tid=UA-26573244-1&cid=1665569122.1508777325&jid=563438466&_gid=738995134.1508777325&gjid=207446299&_v=j64&z=171347024 https://www.google.com/ads/ga-audiences?v=1&aip=1&t=sr&_r=4&tid=UA-26573244-1&cid=1665569122.1508777325&jid=563438466&_v=j64&z=171347024 https://www.google.de/ads/ga-audiences?v=1&aip=1&t=sr&_r=4&tid=UA-26573244-1&cid=1665569122.1508777325&jid=563438466&_v=j64&z=171347024&slf_rd=1&random=2039696993	42 B 60 B	15ms 15ms	Image image/gif	2a00:1450:4001:816::2003 Google Inc.
General Check archive.org Show headers Download Go to Show image Full URL https://www.google.de/ads/ga-audiences?v=1&aip=1&t=sr&_r=4&tid=UA-26573244-1&cid=1665569122.1508777325&jid=563438466&_v=j64&z=171347024&slf_rd=1&random=2039696993 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 2a00:1450:4001:816::2003 , Ireland, ASN15169 (GOOGLE - Google Inc., US), Reverse DNS Software cafe / Resource Hash ef1955ae757c8b966c83248350331bd3a30f658ced11f387f8ebf05ab3368629 Security Headers Name Value X-Content-Type-Options nosniff X-Xss-Protection 1; mode=block Request headers :path /ads/ga-audiences?v=1&aip=1&t=sr&_r=4&tid=UA-26573244-1&cid=1665569122.1508777325&jid=563438466&_v=j64&z=171347024&slf_rd=1&random=2039696993 pragma no-cache accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept image/webp,image/apng,image/*,*/*;q=0.8 cache-control no-cache :authority www.google.de referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers pragma no-cache date Mon, 23 Oct 2017 16:48:45 GMT x-content-type-options nosniff server cafe timing-allow-origin * p3p policyref="https://www.googleadservices.com/pagead/p3p.xml", CP="NOI DEV PSA PSD IVA IVD OTP OUR OTR IND OTC" status 200 cache-control no-cache, must-revalidate content-type image/gif alt-svc quic=":443"; ma=2592000; v="39,38,37,35" content-length 42 x-xss-protection 1; mode=block expires Fri, 01 Jan 1990 00:00:00 GMT Redirect headers pragma no-cache date Mon, 23 Oct 2017 16:48:45 GMT x-content-type-options nosniff server cafe p3p policyref="https://www.googleadservices.com/pagead/p3p.xml", CP="NOI DEV PSA PSD IVA IVD OTP OUR OTR IND OTC" status 302 content-type text/html; charset=UTF-8 location https://www.google.de/ads/ga-audiences?v=1&aip=1&t=sr&_r=4&tid=UA-26573244-1&cid=1665569122.1508777325&jid=563438466&_v=j64&z=171347024&slf_rd=1&random=2039696993 cache-control no-cache, must-revalidate timing-allow-origin * alt-svc quic=":443"; ma=2592000; v="39,38,37,35" content-length 0 x-xss-protection 1; mode=block expires Fri, 01 Jan 1990 00:00:00 GMT
GET H2	200	adsct t.co/i/	43 B 74 B	131ms 113ms	Image image/gif	104.244.42.133 Twitter Inc.
General Check archive.org Show headers Download Go to Show image Full URL https://t.co/i/adsct?txn_id=nuqqc&p_id=Twitter&tw_sale_amount=0&tw_order_quantity=0 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 104.244.42.133 San Francisco, United States, ASN13414 (TWITTER - Twitter Inc., US), Reverse DNS Software tsa_o / Resource Hash ac8778041fdb7f2e08ceb574c9a766247ea26f1a7d90fa854c4efcf4b361a957 Security Headers Name Value Strict-Transport-Security max-age=0 X-Content-Type-Options nosniff X-Frame-Options SAMEORIGIN X-Xss-Protection 1; mode=block Request headers :path /i/adsct?txn_id=nuqqc&p_id=Twitter&tw_sale_amount=0&tw_order_quantity=0 pragma no-cache accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept image/webp,image/apng,image/*,*/*;q=0.8 cache-control no-cache :authority t.co referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:45 GMT content-encoding gzip x-content-type-options nosniff status 200 200 OK x-twitter-response-tags BouncerCompliant content-length 65 x-xss-protection 1; mode=block x-response-time 108 pragma no-cache last-modified Mon, 23 Oct 2017 16:48:45 GMT server tsa_o x-frame-options SAMEORIGIN strict-transport-security max-age=0 content-type image/gif;charset=utf-8 cache-control no-cache, no-store, must-revalidate, pre-check=0, post-check=0 x-connection-hash 6b5ede4ab70ee1f9e8567afa2e94ae08 x-transaction 0009ab6a00f43ad8 expires Tue, 31 Mar 1981 05:00:00 GMT
GET		ect.html b.company-target.com/ Frame 2130	0 0
GET H2	200	pixel d.company-target.com/ul_cb/ Redirect Chain https://d.company-target.com/pixel?type=js&id=1501520880&page=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3 https://d.company-target.com/ul_cb/pixel?type=js&id=1501520880&page=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3	421 B 0	39ms 39ms	Image text/javascript	35.190.27.37 Google Inc.
General Check archive.org Show headers Download Go to Show image Full URL https://d.company-target.com/ul_cb/pixel?type=js&id=1501520880&page=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 35.190.27.37 Mountain View, United States, ASN15169 (GOOGLE - Google Inc., US), Reverse DNS 37.27.190.35.bc.googleusercontent.com Software / Resource Hash e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 Request headers :path /ul_cb/pixel?type=js&id=1501520880&page=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3 pragma no-cache cookie tuuid=636508ac-198b-4e5f-bf05-c319cdda9e77; tuuid_last_update=1508777325 accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept image/webp,image/apng,image/*,*/*;q=0.8 cache-control no-cache :authority d.company-target.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:45 GMT via 1.1 google p3p CP="NOI DSP COR NID CURa ADMa DEVa PSAa PSDa OUR BUS COM INT OTC PUR STA" status 200 cache-control no-cache, no-store, must-revalidate set-cookie tuuid=636508ac-198b-4e5f-bf05-c319cdda9e77; path=/; expires=Wed, 23-Oct-2019 16:48:45 GMT; domain=.company-target.com tuuid_last_update=1508777325; path=/; expires=Wed, 23-Oct-2019 16:48:45 GMT; domain=.company-target.com content-type text/javascript; charset=UTF-8 alt-svc clear content-length 421 Redirect headers date Mon, 23 Oct 2017 16:48:45 GMT via 1.1 google status 302 p3p CP="NOI DSP COR NID CURa ADMa DEVa PSAa PSDa OUR BUS COM INT OTC PUR STA" location https://d.company-target.com/ul_cb/pixel?type=js&id=1501520880&page=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3 cache-control no-cache, no-store, must-revalidate set-cookie tuuid=636508ac-198b-4e5f-bf05-c319cdda9e77; path=/; expires=Wed, 23-Oct-2019 16:48:45 GMT; domain=.company-target.com tuuid_last_update=1508777325; path=/; expires=Wed, 23-Oct-2019 16:48:45 GMT; domain=.company-target.com alt-svc clear content-length 0
GET H2	200	pixel d.company-target.com/ul_cb/ Redirect Chain https://d.company-target.com/pixel?type=js&id=1501520919&page=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3 https://d.company-target.com/ul_cb/pixel?type=js&id=1501520919&page=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3	421 B 0	16ms 16ms	Image text/javascript	35.190.27.37 Google Inc.
General Check archive.org Show headers Download Go to Show image Full URL https://d.company-target.com/ul_cb/pixel?type=js&id=1501520919&page=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3 Protocol H2 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 35.190.27.37 Mountain View, United States, ASN15169 (GOOGLE - Google Inc., US), Reverse DNS 37.27.190.35.bc.googleusercontent.com Software / Resource Hash e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 Request headers :path /ul_cb/pixel?type=js&id=1501520919&page=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3 pragma no-cache cookie tuuid=f9bf39c9-adc6-45c4-8678-2c8dfdf1a4cc; tuuid_last_update=1508777325 accept-encoding gzip, deflate user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 accept image/webp,image/apng,image/*,*/*;q=0.8 cache-control no-cache :authority d.company-target.com referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 :scheme https :method GET Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Response headers date Mon, 23 Oct 2017 16:48:45 GMT via 1.1 google p3p CP="NOI DSP COR NID CURa ADMa DEVa PSAa PSDa OUR BUS COM INT OTC PUR STA" status 200 cache-control no-cache, no-store, must-revalidate set-cookie tuuid=f9bf39c9-adc6-45c4-8678-2c8dfdf1a4cc; path=/; expires=Wed, 23-Oct-2019 16:48:45 GMT; domain=.company-target.com tuuid_last_update=1508777325; path=/; expires=Wed, 23-Oct-2019 16:48:45 GMT; domain=.company-target.com content-type text/javascript; charset=UTF-8 alt-svc clear content-length 421 Redirect headers date Mon, 23 Oct 2017 16:48:45 GMT via 1.1 google status 302 p3p CP="NOI DSP COR NID CURa ADMa DEVa PSAa PSDa OUR BUS COM INT OTC PUR STA" location https://d.company-target.com/ul_cb/pixel?type=js&id=1501520919&page=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3 cache-control no-cache, no-store, must-revalidate set-cookie tuuid=f9bf39c9-adc6-45c4-8678-2c8dfdf1a4cc; path=/; expires=Wed, 23-Oct-2019 16:48:45 GMT; domain=.company-target.com tuuid_last_update=1508777325; path=/; expires=Wed, 23-Oct-2019 16:48:45 GMT; domain=.company-target.com alt-svc clear content-length 0
GET H/1.1	200 OK	ip.json Show response api.demandbase.com/api/v2/	1 KB 584 B	81ms 54ms	XHR application/json	54.230.14.50 Amazon.com
General Check archive.org Show headers Download Go to Full URL https://api.demandbase.com/api/v2/ip.json?referrer=&page=https%3A%2F%2Fwww.digitalocean.com%2Fcommunity%2Ftutorials%2Fhow-to-crawl-a-web-page-with-scrapy-and-python-3&page_title=Crawling%20and%20Scraping%20Web%20Pages%20with%20Scrapy%20and%20Python%203%20%7C%20DigitalOcean&key=68d58b97378ff8978297ffef7277e829 Requested by Host: scripts.demandbase.com URL: https://scripts.demandbase.com/014ab3bd.min.js Protocol HTTP/1.1 Security TLS 1.2, ECDHE_RSA, AES_128_GCM Server 54.230.14.50 Seattle, United States, ASN16509 (AMAZON-02 - Amazon.com, Inc., US), Reverse DNS server-54-230-14-50.ams1.r.cloudfront.net Software nginx / Resource Hash 8a9ca11a8a9d245c5432950179954a21503ff248a0a68bfa8ef3013cb2cda5b9 Request headers Pragma no-cache Origin https://www.digitalocean.com Accept-Encoding gzip, deflate Host api.demandbase.com User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Accept */* Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Connection keep-alive Cache-Control no-cache User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/62.0.3202.62 Safari/537.36 Referer https://www.digitalocean.com/community/tutorials/how-to-crawl-a-web-page-with-scrapy-and-python-3 Origin https://www.digitalocean.com Response headers Date Mon, 23 Oct 2017 16:48:49 GMT Content-Encoding gzip Access-Control-Allow-Origin https://www.digitalocean.com X-Cache Miss from cloudfront Access-Control-Max-Age 1728000 Connection keep-alive Request-ID 7421c5bd-649b-456e-917e-c5716500ce79 Content-Length 584 Pragma no-cache Server nginx Vary Accept-Encoding, Origin Access-Control-Allow-Methods GET, POST, PUT, DELETE, OPTIONS Content-Type application/json;charset=utf-8 Via 1.1 d9552fc5d203b7c80e0dc882434351b8.cloudfront.net (CloudFront) Cache-Control no-cache, no-store, max-age=0, must-revalidate Access-Control-Allow-Credentials true Api-Version v2 Access-Control-Allow-Headers DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type X-Amz-Cf-Id JGkopAuSfl6Mg9XyzKTYGaRbdzhwyMftNRyrYzMl4ihWSVETvUE_Bg== Expires Sun, 22 Oct 2017 16:48:49 GMT

Failed requests

These URLs were requested, but there was no response received. You will also see them in the list above.

Domain

platform.twitter.com

URL

https://platform.twitter.com/widgets/twitter_cookies.html?namespace=twttr%3Acookies&origin=https%3A%2F%2Fwww.digitalocean.com

Domain

staticxx.facebook.com

URL

https://staticxx.facebook.com/connect/xd_arbiter/r/hsBwMj6iLmk.js?version=42

Domain

staticxx.facebook.com

URL

https://staticxx.facebook.com/connect/xd_arbiter/r/hsBwMj6iLmk.js?version=42

Domain

b.company-target.com

URL

https://b.company-target.com/ect.html