Submitted URL: https://www.searchenginejournal.com/x-robots-tag-simple-alternate-robots-txt-meta-tag/67138/
Effective URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Submission: On December 15 via api from GB — Scanned from GB

Summary

This website contacted 28 IPs in 4 countries across 20 domains to perform 138 HTTP transactions. The main IP is 136.243.171.225, located in Ludwigshafen am Rhein, Germany and belongs to HETZNER-AS, DE. The main domain is www.searchenginejournal.com. The Cisco Umbrella rank of the primary domain is 180633.
TLS certificate: Issued by R3 on November 2nd 2023. Valid for: 3 months.
This is the only time www.searchenginejournal.com was scanned on urlscan.io!

urlscan.io Verdict: No classification

Domain & IP information

IP Address AS Autonomous System
2 90 136.243.171.225 24940 (HETZNER-AS)
9 2606:4700:303... 13335 (CLOUDFLAR...)
2 2a00:1450:400... 15169 (GOOGLE)
1 2606:4700:10:... 13335 (CLOUDFLAR...)
3 99.84.149.113 16509 (AMAZON-02)
1 104.237.62.212 18450 (WEBNX)
3 130.211.23.194 396982 (GOOGLE-CL...)
2 2606:4700:20:... 13335 (CLOUDFLAR...)
1 142.250.181.230 15169 (GOOGLE)
1 99.86.4.128 16509 (AMAZON-02)
1 13.227.219.68 16509 (AMAZON-02)
1 184.30.211.26 16625 (AKAMAI-AS)
1 18.64.79.46 16509 (AMAZON-02)
1 2606:4700:10:... 13335 (CLOUDFLAR...)
1 2606:4700:10:... 13335 (CLOUDFLAR...)
2 2606:4700:10:... 13335 (CLOUDFLAR...)
1 18.64.119.20 16509 (AMAZON-02)
2 54.216.91.216 16509 (AMAZON-02)
1 13.32.99.97 16509 (AMAZON-02)
1 2606:4700:10:... 13335 (CLOUDFLAR...)
5 2a00:1450:400... 15169 (GOOGLE)
1 2600:9000:226... 16509 (AMAZON-02)
2 2001:4860:480... 15169 (GOOGLE)
2 2a00:1450:400... 15169 (GOOGLE)
2 2a00:1450:400... 15169 (GOOGLE)
2 2001:4860:480... 15169 (GOOGLE)
1 18.64.103.125 16509 (AMAZON-02)
138 28
Apex Domain
Subdomains
Transfer
90 searchenginejournal.com
www.searchenginejournal.com — Cisco Umbrella Rank: 180633
2 MB
9 uniconsent.com
cmp.uniconsent.com — Cisco Umbrella Rank: 27658
159 KB
5 googletagmanager.com
www.googletagmanager.com — Cisco Umbrella Rank: 36
455 KB
5 doubleclick.net
securepubads.g.doubleclick.net — Cisco Umbrella Rank: 196
ad.doubleclick.net — Cisco Umbrella Rank: 139
stats.g.doubleclick.net — Cisco Umbrella Rank: 75
165 KB
4 amazon-adsystem.com
c.amazon-adsystem.com — Cisco Umbrella Rank: 306
config.aps.amazon-adsystem.com — Cisco Umbrella Rank: 614
78 KB
4 btloader.com
btloader.com — Cisco Umbrella Rank: 931
api.btloader.com — Cisco Umbrella Rank: 1000
18 KB
3 ad.gt
id.hadron.ad.gt — Cisco Umbrella Rank: 1673
a.ad.gt — Cisco Umbrella Rank: 1869
4 KB
3 crwdcntrl.net
tags.crwdcntrl.net — Cisco Umbrella Rank: 979
bcp.crwdcntrl.net — Cisco Umbrella Rank: 850
id.crwdcntrl.net — Cisco Umbrella Rank: 2417
13 KB
2 google-analytics.com
region1.google-analytics.com — Cisco Umbrella Rank: 2189
299 B
2 google.co.uk
www.google.co.uk — Cisco Umbrella Rank: 3754
515 B
2 google.com
region1.analytics.google.com — Cisco Umbrella Rank: 2693
308 B
2 lfeeder.com
sc.lfeeder.com — Cisco Umbrella Rank: 15381
tr.lfeeder.com — Cisco Umbrella Rank: 25528
11 KB
2 ad-delivery.net
ad-delivery.net — Cisco Umbrella Rank: 1018
1 KB
1 openfpcdn.io
openfpcdn.io — Cisco Umbrella Rank: 15226
5 KB
1 privacymanager.io
geo.privacymanager.io — Cisco Umbrella Rank: 2070
609 B
1 id5-sync.com
cdn.id5-sync.com — Cisco Umbrella Rank: 893
33 KB
1 hadronid.net
cdn.hadronid.net — Cisco Umbrella Rank: 1790
10 KB
1 fastclick.net
secure.cdn.fastclick.net — Cisco Umbrella Rank: 1042
17 KB
1 rlcdn.com
ats.rlcdn.com — Cisco Umbrella Rank: 4480
35 KB
1 ipify.org
api.ipify.org — Cisco Umbrella Rank: 2843
224 B
138 20
Domain Requested by
90 www.searchenginejournal.com 2 redirects www.searchenginejournal.com
9 cmp.uniconsent.com www.searchenginejournal.com
cmp.uniconsent.com
5 www.googletagmanager.com www.searchenginejournal.com
www.googletagmanager.com
3 api.btloader.com btloader.com
3 c.amazon-adsystem.com www.searchenginejournal.com
c.amazon-adsystem.com
2 region1.google-analytics.com www.googletagmanager.com
2 www.google.co.uk
2 stats.g.doubleclick.net www.googletagmanager.com
2 region1.analytics.google.com www.googletagmanager.com
2 id.hadron.ad.gt cdn.hadronid.net
2 ad-delivery.net www.searchenginejournal.com
2 securepubads.g.doubleclick.net www.searchenginejournal.com
securepubads.g.doubleclick.net
1 id.crwdcntrl.net www.searchenginejournal.com
1 tr.lfeeder.com
1 sc.lfeeder.com www.searchenginejournal.com
1 a.ad.gt cdn.hadronid.net
1 openfpcdn.io www.searchenginejournal.com
1 bcp.crwdcntrl.net tags.crwdcntrl.net
1 geo.privacymanager.io ats.rlcdn.com
1 cdn.id5-sync.com www.searchenginejournal.com
1 cdn.hadronid.net www.searchenginejournal.com
1 tags.crwdcntrl.net www.searchenginejournal.com
1 secure.cdn.fastclick.net www.searchenginejournal.com
1 ats.rlcdn.com www.searchenginejournal.com
1 config.aps.amazon-adsystem.com c.amazon-adsystem.com
1 ad.doubleclick.net www.searchenginejournal.com
1 api.ipify.org www.searchenginejournal.com
1 btloader.com www.searchenginejournal.com
138 28
Subject Issuer Validity Valid
www.searchenginejournal.com
R3
2023-11-02 -
2024-01-31
3 months crt.sh
uniconsent.com
GTS CA 1P5
2023-11-10 -
2024-02-08
3 months crt.sh
*.g.doubleclick.net
GTS CA 1C3
2023-11-20 -
2024-02-12
3 months crt.sh
btloader.com
GTS CA 1P5
2023-10-19 -
2024-01-17
3 months crt.sh
c.amazon-adsystem.com
Amazon RSA 2048 M01
2023-02-28 -
2024-02-17
a year crt.sh
*.ipify.org
Sectigo RSA Domain Validation Secure Server CA
2023-02-07 -
2024-02-18
a year crt.sh
api.btloader.com
GTS CA 1D4
2023-12-08 -
2024-03-07
3 months crt.sh
sni.cloudflaressl.com
Cloudflare Inc ECC CA-3
2023-02-19 -
2024-02-19
a year crt.sh
*.doubleclick.net
GTS CA 1C3
2023-11-20 -
2024-02-12
3 months crt.sh
config.aps.amazon-adsystem.com
Amazon RSA 2048 M02
2023-02-20 -
2024-03-20
a year crt.sh
*.rlcdn.com
Sectigo RSA Domain Validation Secure Server CA
2023-02-02 -
2024-03-03
a year crt.sh
secure.cdn.fastclick.net
DigiCert TLS RSA SHA256 2020 CA1
2023-10-03 -
2024-10-03
a year crt.sh
*.crwdcntrl.net
Amazon RSA 2048 M01
2023-10-08 -
2024-11-05
a year crt.sh
hadronid.net
GTS CA 1P5
2023-12-03 -
2024-03-02
3 months crt.sh
*.privacymanager.io
Amazon RSA 2048 M01
2023-07-27 -
2024-08-24
a year crt.sh
openfpcdn.io
Amazon RSA 2048 M01
2023-01-25 -
2024-02-24
a year crt.sh
a.ad.gt
E1
2023-12-12 -
2024-03-11
3 months crt.sh
*.google-analytics.com
GTS CA 1C3
2023-11-20 -
2024-02-12
3 months crt.sh
*.lfeeder.com
Amazon RSA 2048 M01
2023-03-22 -
2024-04-19
a year crt.sh
www.google.co.uk
GTS CA 1C3
2023-11-20 -
2024-02-12
3 months crt.sh

This page contains 1 frames:

Primary Page: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Frame ID: 863268377C63128233A861F3D371A58A
Requests: 152 HTTP requests in this frame

Screenshot

Page Title

Best Practices for Setting Up Meta Robots Tags & Robots.txt

Page URL History Show full URLs

  1. https://www.searchenginejournal.com/x-robots-tag-simple-alternate-robots-txt-meta-tag/67138/ HTTP 301
    https://www.searchenginejournal.com/best-practices-setting-meta-robots-tags-robots-txt/188655/ HTTP 301
    https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/ Page URL

Detected technologies

Overall confidence: 100%
Detected patterns
  • /wp-(?:content|includes)/

Overall confidence: 100%
Detected patterns

Overall confidence: 100%
Detected patterns
  • googletagmanager\.com/gtm\.js
  • googletagmanager\.com/gtag/js


Overall confidence: 100%
Detected patterns
  • jquery[.-]([\d.]*\d)[^/]*\.js
  • jquery.*\.js(?:\?ver(?:sion)?=([\d.]+))?

Page Statistics

138
Requests

100 %
HTTPS

52 %
IPv6

20
Domains

28
Subdomains

28
IPs

4
Countries

3295 kB
Transfer

7184 kB
Size

13
Cookies

Page URL History

This captures the URL locations of the websites, including HTTP redirects and client-side redirects via JavaScript or Meta fields.

  1. https://www.searchenginejournal.com/x-robots-tag-simple-alternate-robots-txt-meta-tag/67138/ HTTP 301
    https://www.searchenginejournal.com/best-practices-setting-meta-robots-tags-robots-txt/188655/ HTTP 301
    https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/ Page URL

Redirected requests

There were HTTP redirect chains for the following requests:

138 HTTP transactions

Resource
Path
Size
x-fer
Type
MIME-Type
Primary Request /
www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Redirect Chain
  • https://www.searchenginejournal.com/x-robots-tag-simple-alternate-robots-txt-meta-tag/67138/
  • https://www.searchenginejournal.com/best-practices-setting-meta-robots-tags-robots-txt/188655/
  • https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
390 KB
77 KB
Document
General
Full URL
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
cfc67cc35c76c672a4b35e86146752a3d437c8e6cf82309d21ffd4c91cb4fd7c
Security Headers
Name Value
Content-Security-Policy upgrade-insecure-requests
Strict-Transport-Security max-age=31536000; preload
X-Content-Type-Options nosniff
X-Frame-Options SAMEORIGIN
X-Xss-Protection 1; mode=block

Request headers

Upgrade-Insecure-Requests
1
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36
accept-language
en-GB,en;q=0.9

Response headers

content-encoding
gzip
content-security-policy
upgrade-insecure-requests
content-type
text/html; charset=UTF-8
date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Fri, 08 May 2020 18:14:19 GMT
referrer-policy
strict-origin-when-cross-origin
server
nginx
strict-transport-security
max-age=31536000; preload
vary
Accept-Encoding, Cookie
x-content-type-options
nosniff
x-frame-options
SAMEORIGIN
x-presslabs-stats
HIT; 0.312s; 15 queries; desktop; ttl 172800s; refresh in 161058s
x-request-id
d28ec77f03d86dec7a8fa077d72c0aee
x-xss-protection
1; mode=block

Redirect headers

cache-control
no-cache, must-revalidate, max-age=0
content-type
text/html; charset=UTF-8
date
Fri, 15 Dec 2023 14:48:17 GMT
expires
Wed, 11 Jan 1984 05:00:00 GMT
location
/technical-seo/meta-robots-tags-robots-txt/
server
nginx
vary
Accept-Encoding
x-presslabs-stats
EXPIRED; 0.175s; 4 queries; desktop; ttl 7200s; refresh in 7200s
x-redirect-by
redirection
x-request-id
98239dee0f2fd47da04b1080fe963490
tcfapi.js
www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/ga/js/
5 KB
2 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/ga/js/tcfapi.js?ver=12
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
e7c302b5ae8fea99e6432528806e19f8a6fce4ab1c386f58f4daf8d10dba6d7a

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 14 Dec 2023 22:33:08 GMT
server
nginx
etag
W/"657b82a4-13bb"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
c002dcaa0fe455e24d03145d93258c52
expires
Thu, 31 Dec 2037 23:55:55 GMT
cmp.js
cmp.uniconsent.com/v2/7d36e67838/
3 KB
1 KB
Script
General
Full URL
https://cmp.uniconsent.com/v2/7d36e67838/cmp.js
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2606:4700:3031::ac43:d3be , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
c7de47a055d9437f1e8ba6b9a6f2c33def0c2f49614305cf849d274a93c59f65

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
br
cf-cache-status
HIT
nel
{"success_fraction":0,"report_to":"cf-nel","max_age":604800}
age
57245
x-cache-status
MISS
alt-svc
h3=":443"; ma=86400
last-modified
Sun, 12 Nov 2023 00:28:55 GMT
server
cloudflare
etag
W/"9a6fdc84d7daadd61af60057f086f03b"
vary
Accept-Encoding
report-to
{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=HcqRG8pikyaPvOW0WfkWA9rKQsLgJirh2T4cNUtC85ojwn8tHsBbc8irvIgedswuduqIaJ5ZUwJFgj3ACELwjJUM8Ar811pKT4kRsAIWpEHrp5G2l9rTwys%2FksTD2ZgnPsZ6sy9uj0Ud1Z3NBEXC%2F4M%3D"}],"group":"cf-nel","max_age":604800}
content-type
application/javascript; charset=utf-8
access-control-allow-origin
*
cache-control
max-age=86400
cf-ray
835f7c9538b5772b-LHR
inter.woff2
www.searchenginejournal.com/wp-content/themes/sej/fonts/
37 KB
37 KB
Font
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/fonts/inter.woff2
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
450f3ba4e47ee174bd9692b396f264b907d37d2528f53911760f3d0edb785f7e

Request headers

Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Origin
https://www.searchenginejournal.com
accept-language
en-GB,en;q=0.9
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
"65718a1e-9424"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
font/woff2
cache-control
max-age = 315360000
accept-ranges
bytes
x-cache-groups
assets
content-length
37924
x-request-id
c002dcaa0fe455e24d03145d93258c52
expires
Thu, 31 Dec 2037 23:55:55 GMT
lora-italic.woff2
www.searchenginejournal.com/wp-content/themes/sej/fonts/
38 KB
39 KB
Font
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/fonts/lora-italic.woff2
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
5d55ce9c3ac7a5f37a38813a2deed310abde9e63d0d36912e2e5e59751ca66c7

Request headers

Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Origin
https://www.searchenginejournal.com
accept-language
en-GB,en;q=0.9
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
"65718a1e-9928"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
font/woff2
cache-control
max-age = 315360000
accept-ranges
bytes
x-cache-groups
assets
content-length
39208
x-request-id
c002dcaa0fe455e24d03145d93258c52
expires
Thu, 31 Dec 2037 23:55:55 GMT
lora-normal.woff2
www.searchenginejournal.com/wp-content/themes/sej/fonts/
35 KB
35 KB
Font
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/fonts/lora-normal.woff2
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
9f46649ea544819982ea288c6f386dd67d46da0f453f95da542196372b79731e

Request headers

Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Origin
https://www.searchenginejournal.com
accept-language
en-GB,en;q=0.9
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
"65718a1e-8c30"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
font/woff2
cache-control
max-age = 315360000
accept-ranges
bytes
x-cache-groups
assets
content-length
35888
x-request-id
c002dcaa0fe455e24d03145d93258c52
expires
Thu, 31 Dec 2037 23:55:55 GMT
location.json
www.searchenginejournal.com/
20 B
155 B
XHR
General
Full URL
https://www.searchenginejournal.com/location.json
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
d632f1fd8fd7e6f6a9fbbdf2ad865c79432d41f8e203db015dfb473005243d95

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
server
nginx
x-presslabs-stats
desktop
vary
Accept-Encoding
x-request-id
c4a7877cacb52132de5b265addb86e21
content-type
application/json
wfs-form-skin.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
58 KB
11 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/wfs-form-skin.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
013eec022b83c1f01c2127108b2724d94f43a2c91ac337a3d317632739117ff0

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-e692"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
c293e2e05f19af5ef3dca054f1a7b48a
expires
Thu, 31 Dec 2037 23:55:55 GMT
wsf-form-classes.css
www.searchenginejournal.com/wp-content/themes/sej/css/abstracts/
5 KB
2 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/abstracts/wsf-form-classes.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
360ec0a5e7fd9eb35367277e9257aa6378fedb7bbf0bc53ad8e3886820086b09

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-15e0"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
c293e2e05f19af5ef3dca054f1a7b48a
expires
Thu, 31 Dec 2037 23:55:55 GMT
variables.css
www.searchenginejournal.com/wp-content/themes/sej/css/abstracts/
1 KB
733 B
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/abstracts/variables.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
9ff3d76471ddf3e6dc85d985ac7911883108c186d71bca9fdaf4524de72ed421

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-440"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
c293e2e05f19af5ef3dca054f1a7b48a
expires
Thu, 31 Dec 2037 23:55:55 GMT
reset.css
www.searchenginejournal.com/wp-content/themes/sej/css/vendor/
4 KB
2 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/vendor/reset.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
eef945f7b56946036572f7b2f4067b0c3c139440720704f39f1e25e7e250a57f

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-ec1"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
c293e2e05f19af5ef3dca054f1a7b48a
expires
Thu, 31 Dec 2037 23:55:55 GMT
bootstrap-grid-v2.css
www.searchenginejournal.com/wp-content/themes/sej/css/vendor/
6 KB
954 B
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/vendor/bootstrap-grid-v2.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
494e6ad3ea844634436d169312c020a27761bd7fc0ae20649624296655c41cfb

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-1894"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
c293e2e05f19af5ef3dca054f1a7b48a
expires
Thu, 31 Dec 2037 23:55:55 GMT
typography.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
5 KB
2 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/typography.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
111d4518d77b165368c37669f563a1d435e3766dda9544ae28c7ed965e05afb6

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-1318"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
c293e2e05f19af5ef3dca054f1a7b48a
expires
Thu, 31 Dec 2037 23:55:55 GMT
layout.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
2 KB
1 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/layout.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
0c9fb498a29dd3d10f8a6ade5bfb1b38aa0f91be670eb0c538413963d1bc06ac

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-88c"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
c293e2e05f19af5ef3dca054f1a7b48a
expires
Thu, 31 Dec 2037 23:55:55 GMT
navbar-desktop.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
11 KB
3 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/navbar-desktop.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
ee7300345a6db37b152f33f268dbc82862cc88242fa6ccb7f0b4b35bc350fecd

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-2b84"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
c293e2e05f19af5ef3dca054f1a7b48a
expires
Thu, 31 Dec 2037 23:55:55 GMT
buttons.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
3 KB
1 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/buttons.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
63fa00bcc8093c9fc6652d02aebfe2e54e99ebad34954f877bdb51e655bdc5e2

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-a93"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
c293e2e05f19af5ef3dca054f1a7b48a
expires
Thu, 31 Dec 2037 23:55:55 GMT
inputs.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
4 KB
2 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/inputs.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
c626dabe59f1e78ba16b99eec8c8291f67ce15788b7556911f68b9be562ac5bb

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-f64"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
navbar-more.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
1 KB
768 B
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/navbar-more.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
1d457324cbe10e73b26ccfbbe9b797e6972b85d4ba1419c4ab233b0ec427fcc1

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-484"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
sidebar.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
2 KB
966 B
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/sidebar.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
8b9180ed5e3b5e2d1acd5e2215070d59a589eae5de6ccd840e9e700acb71dcec

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-6a1"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
breadcrumbs.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
476 B
545 B
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/breadcrumbs.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
8af8ed19513c17a73f34a27b342003a52850988e6aa67c97c808f340d5f09878

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-1dc"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
chapters.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
2 KB
1003 B
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/chapters.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
4fca7e9d2305082e22ac89090f8b7e9488602b6b4ef27dd30c9342f9c36b0c1e

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-7fa"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
faq.css
www.searchenginejournal.com/wp-content/themes/sej/css/pages/
718 B
592 B
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/pages/faq.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
2bc0406454463b7cca251018ae904ae21c02f0e835acd296c5a8229074756245

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-2ce"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
fancybox.css
www.searchenginejournal.com/wp-content/themes/sej/css/vendor/
23 KB
6 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/vendor/fancybox.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
46e1a5317649fd50f6f2cd67bbb8efb9a176e613d75442987855ba686b7682bf

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-5cd2"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
owl.carousel.css
www.searchenginejournal.com/wp-content/themes/sej/css/vendor/
3 KB
1 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/vendor/owl.carousel.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
1b9d62fca3818f9b4d6fbe625e0b7adeb32c98a9eac289519f6bf7c165571c12

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-c65"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
article.css
www.searchenginejournal.com/wp-content/themes/sej/css/pages/
23 KB
7 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/pages/article.css?ver=1702391798
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
2868c587f51b3ff82a480f29218ce4b911a1cea8e4f2815ff5112ff56bb4362e

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Tue, 12 Dec 2023 14:36:38 GMT
server
nginx
etag
W/"65786ff6-5a6c"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
fancy-theme.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
3 KB
1 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/fancy-theme.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
57b19a7a79fcf081749d34fdd3a96cbc11419416b760a3be213aaf509097de17

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-a7b"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
ifrmae-css.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
722 B
491 B
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/ifrmae-css.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
340a49ba6b22c47b01b4aa9e42bc9b6b75f0a3b7938724ccf9347579cd0a0668

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-2d2"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
share-buttons.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
3 KB
1 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/share-buttons.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
f50db6abd4c6f1298f381d938cee3dbc3fc2efd813723b5cbe62e2b215f036cc

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-aad"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
footer.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
3 KB
1 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/footer.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
34584471249c18eec831bf83fa69311328233f05a87e909ccba4d9623620a1db

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-a54"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
global-non-critical.css
www.searchenginejournal.com/wp-content/themes/sej/css/abstracts/
2 KB
935 B
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/abstracts/global-non-critical.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
5da4ec024f5d8814b0f272f9e9ba8457e0c1615c7e25a4fda2dcf550eb46efe3

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-60c"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
public.layout.min.css
www.searchenginejournal.com/wp-content/uploads/ws-form/css/public/
16 KB
2 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/ws-form/css/public/public.layout.min.css?ver=1701762877
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
6095256b5d03a355c7dd87945ee5022f7a17135e2cba4f28834598cec0b09238

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Tue, 05 Dec 2023 07:54:37 GMT
server
nginx
etag
W/"656ed73d-3f48"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
utilities.css
www.searchenginejournal.com/wp-content/themes/sej/css/abstracts/
8 KB
3 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/abstracts/utilities.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
c84978335d23fa8e42836a32463987ccff0e9ebe9ad4429235f6a6da9bf19a96

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-1f2b"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
classes.css
www.searchenginejournal.com/wp-content/themes/sej/css/abstracts/
6 KB
2 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/abstracts/classes.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
404946964b5a4084de4bdbeb860c1679ee825c1436e6269fcf48f889952394f7

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-17f6"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
gpt.js
securepubads.g.doubleclick.net/tag/js/
90 KB
29 KB
Script
General
Full URL
https://securepubads.g.doubleclick.net/tag/js/gpt.js
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2a00:1450:4001:829::2002 Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),
Reverse DNS
Software
cafe /
Resource Hash
1fcc2492b4655a83c76b09d6ad4cab1fc73668994e3f2d01dcfcd384fcfa49ae
Security Headers
Name Value
X-Content-Type-Options nosniff
X-Xss-Protection 0

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
br
x-content-type-options
nosniff
p3p
policyref="https://googleads.g.doubleclick.net/pagead/gcn_p3p_.xml", CP="CURa ADMa DEVa TAIo PSAo PSDo OUR IND UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR"
cross-origin-resource-policy
cross-origin
content-disposition
attachment; filename="f.txt"
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length
29306
x-xss-protection
0
server
cafe
etag
623 / 19706 / 31080057 / config-hash: 17400476758908410755
vary
Accept-Encoding
content-type
text/javascript; charset=UTF-8
cache-control
private, max-age=900, stale-while-revalidate=3600
timing-allow-origin
*
expires
Fri, 15 Dec 2023 14:48:17 GMT
prebid-8.27L.js
www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/infinte-scroll/js/prebid/
420 KB
159 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/infinte-scroll/js/prebid/prebid-8.27L.js
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
4d22abbb9a6a14f22b41d50b8e97dcad4a9289af3df46a615b0c70f586d645a3

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Wed, 13 Dec 2023 15:24:40 GMT
server
nginx
etag
W/"6579ccb8-69142"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
8631726f3b9150093f4df9672b6e8512
expires
Thu, 31 Dec 2037 23:55:55 GMT
tag
btloader.com/
52 KB
18 KB
Script
General
Full URL
https://btloader.com/tag?o=5132149699641344&upapi=true
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2606:4700:10::6816:4bd8 , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
7acca53b6e2ee3f5fcbe256dde41cd1ef8906df5c998c0130830bbb37fe3fbca

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
via
1.1 google
cf-cache-status
HIT
last-modified
Fri, 15 Dec 2023 13:55:35 GMT
server
cloudflare
age
2994
etag
"75133836ae8df1e4ddc5a697ecb1988a"
vary
Origin, Accept-Encoding
content-type
application/javascript
cache-control
public, max-age=300, must-revalidate, stale-if-error=3600, stale-while-revalidate=300
accept-ranges
bytes
cf-ray
835f7c971e7048af-LHR
content-length
18020
hb_js.js
www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/infinte-scroll/hb/
46 KB
12 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/infinte-scroll/hb/hb_js.js?ver=1702638412
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
c8ae451dc07e70ed8d9e22c8a12359517d8eeab96b4b16b036f129a28508a980

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Fri, 15 Dec 2023 11:06:52 GMT
server
nginx
etag
W/"657c334c-b76c"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
d4d55342a70ddf2b930fded703c7b593
expires
Thu, 31 Dec 2037 23:55:55 GMT
observer-lazy-load-a.js
www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/infinte-scroll/js/
18 KB
6 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/infinte-scroll/js/observer-lazy-load-a.js?ver=1702639955
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
1f006bec36d62d033e3a64bd87ec8de0cdff7541f0e6950454d4ef866c042ee6

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 11:50:11 GMT
server
nginx
etag
W/"6571b173-48f2"
x-presslabs-cache
MISS
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
jquery-3.6.1.min.js
www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/jquery/
88 KB
36 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/jquery/jquery-3.6.1.min.js?ver=1701939740
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
a3cf00c109d907e543bc4f6dbc85eb31068f94515251347e9e57509b52ee3d74

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:20 GMT
server
nginx
etag
W/"65718a1c-15e40"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
pix.js
www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/ga/js/
2 KB
1 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/ga/js/pix.js?ver=1701939740
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
6c1056b1b1182ed0a3532a6fdc40375191c21146b8387468c8561e317483395f

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:20 GMT
server
nginx
etag
W/"65718a1c-8f7"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
search-engine-journal-1x-63a425d735aa3-sej.webp
www.searchenginejournal.com/wp-content/uploads/2022/12/
6 KB
6 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2022/12/search-engine-journal-1x-63a425d735aa3-sej.webp
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
98aabe4d6eacbd5bb3bffeeb71a0946e9a7241984ef448a981d8f64618b60453

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Thu, 22 Dec 2022 09:39:36 GMT
server
nginx
etag
"63a425d8-18de"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
6366
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
hello-bar.css
www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/hello-bar/css/
2 KB
1 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/hello-bar/css/hello-bar.css?ver=1701939740
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
c02addcf940dcfd73293ccb530238f1f1737d33a6f61f324ef09ad1c69fcf594

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:20 GMT
server
nginx
etag
W/"65718a1c-9bf"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
hello.js
www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/hello-bar/js/
6 KB
2 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/hello-bar/js/hello.js?ver=1702585237
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
e992ee382f729e845f3808b58e9af88bd5f8a49b2ad49f7eaa4b6da0074c8cb3

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 14 Dec 2023 20:20:37 GMT
server
nginx
etag
W/"657b6395-169c"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
ga-prom.js
www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/promos/assets/
3 KB
1 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/promos/assets/ga-prom.js?ver=1701939740
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
9a1eb559f60a9f2344882c89cc97365185663d3507382c35d77a30ad5bfda98a

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:20 GMT
server
nginx
etag
W/"65718a1c-d73"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
nav-bar-168x48-657820a1b9501-sej.gif
www.searchenginejournal.com/wp-content/uploads/2023/12/
514 KB
515 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2023/12/nav-bar-168x48-657820a1b9501-sej.gif
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
7354cd99c980e411b94fb6ac1531864b79360a6eae38da19300316344c1af71b

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Tue, 12 Dec 2023 08:58:27 GMT
server
nginx
etag
"657820b3-807a3"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/gif
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
526243
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
top-desktop-970x200-1x-65782028c8f0f-sej.webp
www.searchenginejournal.com/wp-content/uploads/2023/12/
66 KB
67 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2023/12/top-desktop-970x200-1x-65782028c8f0f-sej.webp
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
5485428ad75e744e1ca665f207db700ed41a3c0a9244bb52637252a14ae841eb

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Tue, 12 Dec 2023 08:56:10 GMT
server
nginx
etag
"6578202a-108b2"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
67762
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
best-practices-for-setting-up-meta-robots-tags-robots.txt-5e397a5392f8e-760x400.webp
www.searchenginejournal.com/wp-content/uploads/2020/01/
28 KB
28 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/01/best-practices-for-setting-up-meta-robots-tags-robots.txt-5e397a5392f8e-760x400.webp
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
daa5cbff105aa4dd98b3982e70a45b7668f394e78b3e5b8a8a2ed470fa83c9c0

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Mon, 16 Jan 2023 06:23:59 GMT
server
nginx
etag
"63c4ed7f-7018"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
28696
x-request-id
7f1fcde1344bcc3c3ba523b02d190545
expires
Thu, 31 Dec 2037 23:55:55 GMT
content-add.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
929 B
746 B
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/content-add.css?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
e98439db943f5748a9bcc5f687bb3b66b3c50f9b77cf3d4eb4ad2fd53aed1258

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-3a1"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
651a2b6c5fc139d33df3269d87a63e05
expires
Thu, 31 Dec 2037 23:55:55 GMT
frontend.min.js
www.searchenginejournal.com/wp-content/themes/sej/js/
4 KB
2 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/js/frontend.min.js?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
79015be0ad1b860341c8a74cc4311ba5ea9b1502f467dc60b26894bf98b69f45

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-f54"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
fancybox.umd.js
www.searchenginejournal.com/wp-content/themes/sej/js/vendor/
139 KB
49 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/js/vendor/fancybox.umd.js?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
b81b00bc7e76a7bb3b4c63f58208f092f0dee448ef77b1a117822fa9dbcf0e89

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-22b6e"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
owl.carousel.min.js
www.searchenginejournal.com/wp-content/themes/sej/js/vendor/
43 KB
14 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/js/vendor/owl.carousel.min.js?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
1756fb46aa1a7fe26e8c57328db54497ef70236895be1a7160718bb772abe23b

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-ad3a"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
gallery-min.js
www.searchenginejournal.com/wp-content/themes/sej/js/pages/article/
2 KB
1 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/js/pages/article/gallery-min.js?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
9c1a2fb00222ceeef376367653f9de6f386eff7ba9793eec110fd7be28421aca

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-69b"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
instapage.js
www.searchenginejournal.com/wp-content/themes/sej14/js/
4 KB
2 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej14/js/instapage.js?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
c4abf725ca2c888df89bee0a81a50fc7dfc69777cec49ef80ceef97123983f47

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-e64"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
cdet.js
www.searchenginejournal.com/wp-content/themes/sej14/js/
2 KB
1 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej14/js/cdet.js?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
e5429ce4c488099db846eb6568ffd7965bca538d9a28cec723bf561c372880b3

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-82e"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
lazyload.js
www.searchenginejournal.com/wp-content/themes/sej14/js/lib/
13 KB
5 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej14/js/lib/lazyload.js?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
85b0fe439768c208c0b3d9157178b8a1ac48fba8a191a3d8d7f1c3c1dcb1dcbd

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-3467"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
ga.js
www.searchenginejournal.com/wp-content/themes/sej/js/
5 KB
2 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/js/ga.js?ver=1702067863
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
999abb9809fe567abcda124de9ad2962f0f66c67c029ed96c0c629bb4c1d4878

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Fri, 08 Dec 2023 20:37:43 GMT
server
nginx
etag
W/"65737e97-1574"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
sej-new.js
www.searchenginejournal.com/wp-content/themes/sej/js/
27 KB
10 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/js/sej-new.js?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
9fd122da02d8e041cd491f078a0fddb3986846255c5a0fc64cbf37eddaeab0e7

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-6db3"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
ws-form.min.js
www.searchenginejournal.com/wp-content/plugins/ws-form-pro/shared/js/
103 KB
31 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/ws-form-pro/shared/js/ws-form.min.js?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
f203715aaaa622db1449a0b2f5695456d5d116f2464c4b144145f041d4a8b949

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-19a86"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
ws-form-public-checkbox.min.js
www.searchenginejournal.com/wp-content/plugins/ws-form-pro/public/js/
2 KB
1 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/ws-form-pro/public/js/ws-form-public-checkbox.min.js?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
20f405357a6a92ff60139fd91e0dc0c0133d9da8efa70ae530e50e04786667d0

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-812"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
ws-form-public-conditional.min.js
www.searchenginejournal.com/wp-content/plugins/ws-form-pro/public/js/
38 KB
11 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/ws-form-pro/public/js/ws-form-public-conditional.min.js?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
d571826e6090eb9ece1e287d9b726b1f63d2e43a3a84a4d17ce466468d14be6f

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-96aa"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
ws-form-public.min.js
www.searchenginejournal.com/wp-content/plugins/ws-form-pro/public/js/
61 KB
17 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/ws-form-pro/public/js/ws-form-public.min.js?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
1f0c0d5a173bd397b39ee432ffaf04528f62188debc93b819ed6c36521e59b5d

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-f247"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
ws-form-public-tracking.min.js
www.searchenginejournal.com/wp-content/plugins/ws-form-pro/public/js/
2 KB
978 B
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/ws-form-pro/public/js/ws-form-public-tracking.min.js?ver=1701939742
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
3aa68b6ea45ed5e1eaaf779a2377f7689dc531620ee9d57a6c0f6444ad11f7e3

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-6dd"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
disp-txt.js
www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/promos/assets/
1 KB
907 B
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/promos/assets/disp-txt.js?ver=1702638530
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
9fac1c1beffa703d282e759e514fac5f368fbacfb6fedd785c968dd191098bfe

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Fri, 15 Dec 2023 11:08:50 GMT
server
nginx
etag
W/"657c33c2-471"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
1d58517c5ee55ab3e215ffdd6f290564
expires
Thu, 31 Dec 2037 23:55:55 GMT
pop-css.css
www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/promos/assets/
9 KB
3 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/promos/assets/pop-css.css?ver=1702638949
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
977884b3a28fd132dd91138134469596a5af3420ace9799253947e557e4a7483

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Fri, 15 Dec 2023 11:15:49 GMT
server
nginx
etag
W/"657c3565-232b"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
651a2b6c5fc139d33df3269d87a63e05
expires
Thu, 31 Dec 2037 23:55:55 GMT
disp.js
www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/promos/assets/
4 KB
1 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/promos/assets/disp.js?ver=1702638919
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
4b3e23e828653d43cefd0d3b5c4e477c5aa9ab6350e66cdf54d1d658b1ad6db8

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Fri, 15 Dec 2023 11:15:19 GMT
server
nginx
etag
W/"657c3547-fd7"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
dbf537c8f84cda66a7734e7b87d8aaab
expires
Thu, 31 Dec 2037 23:55:55 GMT
pp.js
www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/promos/assets/
23 KB
6 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/promos/assets/pp.js?ver=1701939740
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
72790a1735404379453c3f2c9f095b3c326f10c6efb14485931bac45075afbb9

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:20 GMT
server
nginx
etag
W/"65718a1c-5b0c"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
dbf537c8f84cda66a7734e7b87d8aaab
expires
Thu, 31 Dec 2037 23:55:55 GMT
misc.js
www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/js/
7 KB
3 KB
Script
General
Full URL
https://www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/js/misc.js?ver=1701939740
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
0f8b5718fb74df9076bc80ad9c0f85f387438f22989f6e33cbb5853ec96c6cd7

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:20 GMT
server
nginx
etag
W/"65718a1c-1adf"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
dbf537c8f84cda66a7734e7b87d8aaab
expires
Thu, 31 Dec 2037 23:55:55 GMT
main-v3.min.js
cmp.uniconsent.com/v2/
208 KB
50 KB
Script
General
Full URL
https://cmp.uniconsent.com/v2/main-v3.min.js
Requested by
Host: cmp.uniconsent.com
URL: https://cmp.uniconsent.com/v2/7d36e67838/cmp.js
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2606:4700:3031::ac43:d3be , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
0755f702a81f095a3b3814801f57841ba195a5a0e955b0168ff0a4007127e925

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
br
cf-cache-status
HIT
nel
{"success_fraction":0,"report_to":"cf-nel","max_age":604800}
age
57466
x-cache-status
HIT
alt-svc
h3=":443"; ma=86400
last-modified
Wed, 13 Dec 2023 22:49:26 GMT
server
cloudflare
etag
W/"49fece614c7b3c6c9ed656d6c551280a"
vary
Accept-Encoding
report-to
{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=8DvjebUN1mcGyvkxxiwfRDj4qv7MB%2B3224pmAGstfDdgno38YAxgEJ5ht9qI3mLJAoZ4Zm43tx1tltWoGTXCj1lzzewLjDXdvT3Nu8cTOu2uCrOCwQVOp02kMP53D5bbG6dj%2BepSmP4VpzBP%2BASiEEA%3D"}],"group":"cf-nel","max_age":604800}
content-type
application/javascript; charset=utf-8
access-control-allow-origin
*
cache-control
max-age=86400
cf-ray
835f7c96aa51772b-LHR
apstag.js
c.amazon-adsystem.com/aax2/
282 KB
70 KB
Script
General
Full URL
https://c.amazon-adsystem.com/aax2/apstag.js
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
99.84.149.113 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS
server-99-84-149-113.txl52.r.cloudfront.net
Software
AmazonS3 /
Resource Hash
4d3c300c1cd89393c7f945c06656981e3ac1c034f59996affcd1062a3092f40c

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 13:57:02 GMT
content-encoding
gzip
via
1.1 00746b020527dcdbeca0dab6f6de299a.cloudfront.net (CloudFront), 1.1 f66aaf2d9f6a8e65595a1e24d5f18bcc.cloudfront.net (CloudFront)
last-modified
Tue, 12 Dec 2023 22:20:15 GMT
server
AmazonS3
x-amz-cf-pop
FRA60-P1, TXL52-C1
age
3076
x-amz-server-side-encryption
AES256
etag
W/"d6937d02acbbf691a008906e9d0617e0"
vary
Accept-Encoding
x-cache
Hit from cloudfront
content-type
application/javascript
cache-control
max-age=3600
x-amz-cf-id
pRQbqKqI97lyOdqMoMr6PWbqY1yC5o1g8QnzbcaWLL6HD2-X8EdqqA==
print.css
www.searchenginejournal.com/wp-content/themes/sej/css/components/
5 KB
2 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/themes/sej/css/components/print.css?ver=167
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
3edc0d7881798bbf380ea9dd671c8f90b3e9fd091a73d9fe22c59deb5994ddbe

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-13f6"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
dbf537c8f84cda66a7734e7b87d8aaab
expires
Thu, 31 Dec 2037 23:55:55 GMT
xcs0MhRsfxHG.js
www.searchenginejournal.com/
91 B
302 B
Script
General
Full URL
https://www.searchenginejournal.com/xcs0MhRsfxHG.js?ts=58792
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
8c9bf876364a2e03c7f4e18103df82d248ffbd5bada31d3ecb4fd86936f22fa5

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
server
nginx
x-presslabs-stats
desktop
vary
Accept-Encoding
content-type
application/javascript
cache-control
no-cache
x-request-id
dbf537c8f84cda66a7734e7b87d8aaab
expires
Thu, 01 Jan 1970 00:00:01 GMT
icons.svg
www.searchenginejournal.com/
26 KB
11 KB
Other
General
Full URL
https://www.searchenginejournal.com/icons.svg?ver=138
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
d7f9e18543446ef3f97b128bb12bc097732e3889e98a4fcc6d98801753b4e46d

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-673a"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
image/svg+xml
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
dbf537c8f84cda66a7734e7b87d8aaab
expires
Thu, 31 Dec 2037 23:55:55 GMT
search-engine-journal-1x-63a425d735aa3-sej.webp
www.searchenginejournal.com/wp-content/uploads/2022/12/
6 KB
6 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2022/12/search-engine-journal-1x-63a425d735aa3-sej.webp
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
98aabe4d6eacbd5bb3bffeeb71a0946e9a7241984ef448a981d8f64618b60453

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Thu, 22 Dec 2022 09:39:36 GMT
server
nginx
etag
"63a425d8-18de"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
6366
x-request-id
d4d55342a70ddf2b930fded703c7b593
expires
Thu, 31 Dec 2037 23:55:55 GMT
truncated
/
67 B
0
Image
General
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
486dbb49fb36a8b8e7bb750b738fab6a2bd3aee598b3a71d8da405245447dd0f

Request headers

accept-language
en-GB,en;q=0.9
Referer
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Content-Type
image/svg+xml
truncated
/
68 B
0
Image
General
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
053cb3f560d39447ce5eae1f319e68e76c56c67bcd1727db08c6cd3a480be51f

Request headers

accept-language
en-GB,en;q=0.9
Referer
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Content-Type
image/svg+xml
truncated
/
68 B
0
Image
General
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
a6333929a02027e96ecf154218e5fc09e7e64e629c4284d98766878a09d93988

Request headers

accept-language
en-GB,en;q=0.9
Referer
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Content-Type
image/svg+xml
truncated
/
68 B
0
Image
General
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
ea7f8924b70229fe13fe0587bf81b59fa5a235690c936950f09cd0de156e013e

Request headers

accept-language
en-GB,en;q=0.9
Referer
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Content-Type
image/svg+xml
truncated
/
69 B
0
Image
General
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
4117699ce11d0dab6c19cf61dd4ca27aaa826d2807c3462a624daa20dae58a95

Request headers

accept-language
en-GB,en;q=0.9
Referer
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Content-Type
image/svg+xml
truncated
/
68 B
0
Image
General
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
16abe109580561cc0ef661ba3bc4c5c212998d4f4c9cc0e693e3eba57011e907

Request headers

accept-language
en-GB,en;q=0.9
Referer
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Content-Type
image/svg+xml
truncated
/
68 B
0
Image
General
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
e9dfb181b7284e1d5674c9497da7aa98db42987a389b12f57b344285889bf893

Request headers

accept-language
en-GB,en;q=0.9
Referer
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Content-Type
image/svg+xml
truncated
/
68 B
0
Image
General
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
38c872afb161a43f9bcfad5edefd2d2ee6583001be855fbee5be6069564523bc

Request headers

accept-language
en-GB,en;q=0.9
Referer
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Content-Type
image/svg+xml
truncated
/
68 B
0
Image
General
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
9c2c86e88fe2074f9ddbd073add93e47208ae95d2947deb81e4da32c983b01d5

Request headers

accept-language
en-GB,en;q=0.9
Referer
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Content-Type
image/svg+xml
truncated
/
68 B
0
Image
General
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
fb9cbcc030c3111a1c91b31feab02da45e55a4440b784cb6b4a4f743cd99edf7

Request headers

accept-language
en-GB,en;q=0.9
Referer
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Content-Type
image/svg+xml
truncated
/
69 B
0
Image
General
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
a171e64ec4287990b354e03ec80aa0b83e9c01be406cf9af7c232ec40439e7e5

Request headers

accept-language
en-GB,en;q=0.9
Referer
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Content-Type
image/svg+xml
truncated
/
70 B
0
Image
General
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
53b4ab18872c599c620b3d329d3894f9c2973d3339b4ca4679e6afd4215a01f0

Request headers

accept-language
en-GB,en;q=0.9
Referer
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Content-Type
image/svg+xml
truncated
/
69 B
0
Image
General
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
d7dd0007fc5d64b48c0fd0b26382a8c93b93ca82a9b32a7244cc73f63ba5393c

Request headers

accept-language
en-GB,en;q=0.9
Referer
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Content-Type
image/svg+xml
anna-0015-1-1-5f3edf426b6da-65x65.webp
www.searchenginejournal.com/wp-content/uploads/2020/08/
4 KB
5 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/08/anna-0015-1-1-5f3edf426b6da-65x65.webp
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
917106b3d05f0bda89390297ea5672b298a0bf21bd494a53ba930c235522dd94

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Thu, 20 Aug 2020 20:39:22 GMT
server
nginx
etag
"5f3edf7a-1180"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
4480
x-request-id
dbf537c8f84cda66a7734e7b87d8aaab
expires
Thu, 31 Dec 2037 23:55:55 GMT
icons.svg
www.searchenginejournal.com/
26 KB
11 KB
Other
General
Full URL
https://www.searchenginejournal.com/icons.svg?ver=78
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
d7f9e18543446ef3f97b128bb12bc097732e3889e98a4fcc6d98801753b4e46d

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Thu, 07 Dec 2023 09:02:22 GMT
server
nginx
etag
W/"65718a1e-673a"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
image/svg+xml
cache-control
max-age = 315360000
x-cache-groups
assets
x-request-id
cd9826b97e726f60799235b828534e2b
expires
Thu, 31 Dec 2037 23:55:55 GMT
truncated
/
68 B
0
Image
General
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
0bb059e797c405edc79fe7bc59a52b17a5a3fd3511adaba3a56a1cf543c973aa

Request headers

accept-language
en-GB,en;q=0.9
Referer
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Content-Type
image/svg+xml
public.layout.min.css
www.searchenginejournal.com/wp-content/uploads/ws-form/css/public/
16 KB
2 KB
Stylesheet
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/ws-form/css/public/public.layout.min.css?ver=1701762877
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
6095256b5d03a355c7dd87945ee5022f7a17135e2cba4f28834598cec0b09238

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
gzip
last-modified
Tue, 05 Dec 2023 07:54:37 GMT
server
nginx
etag
W/"656ed73d-3f48"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
text/css
cache-control
max-age = 315360000
x-request-id
cd9826b97e726f60799235b828534e2b
expires
Thu, 31 Dec 2037 23:55:55 GMT
sej-today-1x-644d47cddc24a-sej.webp
www.searchenginejournal.com/wp-content/uploads/2023/04/
3 KB
3 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2023/04/sej-today-1x-644d47cddc24a-sej.webp
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
24ae7b9ac5220f301dcfc8032ba2e78994ef99a7d859c7c343c5829259e318af

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Sat, 29 Apr 2023 16:37:34 GMT
server
nginx
etag
"644d47ce-a22"
x-presslabs-cache
HIT
vary
Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
2594
x-request-id
b431c42488aee76e6a7e11846f111101
expires
Thu, 31 Dec 2037 23:55:55 GMT
/
api.ipify.org/
24 B
224 B
XHR
General
Full URL
https://api.ipify.org/?format=json
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
HTTP/1.1
Security
TLS 1.3, , AES_128_GCM
Server
104.237.62.212 El Segundo, United States, ASN18450 (WEBNX, US),
Reverse DNS
api.ipify.org
Software
nginx/1.25.2 /
Resource Hash
eb76d7c54772efe035bcd270556fc754f63087b700ee4ee5c99ce242f41b9d30

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Access-Control-Allow-Origin
*
Date
Fri, 15 Dec 2023 14:48:18 GMT
Server
nginx/1.25.2
Connection
keep-alive
Content-Length
24
Vary
Origin
Content-Type
application/json
config.json
cmp.uniconsent.com/v2/7d36e67838/
22 KB
4 KB
Fetch
General
Full URL
https://cmp.uniconsent.com/v2/7d36e67838/config.json
Requested by
Host: cmp.uniconsent.com
URL: https://cmp.uniconsent.com/v2/main-v3.min.js
Protocol
H3
Security
QUIC, , AES_128_GCM
Server
2606:4700:3031::ac43:d3be , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
1632154cd5abe8555eb15315d6b3d0b1f40c26d08e6ce78f0160224de320ac4f

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
br
cf-cache-status
DYNAMIC
last-modified
Sun, 12 Mar 2023 03:11:55 GMT
nel
{"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server
cloudflare
etag
W/"6d1f5574b7b8237ab0d95078bdca5c98"
x-cache-status
HIT
report-to
{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=YAt7JI6t%2BmTTxsOkOcZW3I8YN65EKG2sxF%2BybQB0XKkYted5as%2B0AiuRSDZPOGyyKhen4IVX8lc9%2BvbZqztCcBnZUbTC4VZdKN0v1VrUv38of3PD675VC4pbO0CrFUWP17lN1HW9k5%2FQ63J6zi9zqj8%3D"}],"group":"cf-nel","max_age":604800}
content-type
application/json
access-control-allow-origin
*
cache-control
max-age=86400
cf-ray
835f7c977d3263f9-LHR
alt-svc
h3=":443"; ma=86400
robots-txt-file-example-5e31a4d66c64c.png
www.searchenginejournal.com/wp-content/uploads/2020/01/
4 KB
4 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/01/robots-txt-file-example-5e31a4d66c64c.png
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
fb19521eeb71c0b7bcdcbf3f457b81aac65f795eb7bda13b4c305b1f23ffa740

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Wed, 29 Jan 2020 15:38:47 GMT
server
nginx
etag
"5e31a707-101e"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
4126
x-request-id
d09faf5ec4c8bdb6d9a5200c6bf92b3a
expires
Thu, 31 Dec 2037 23:55:55 GMT
moz-robots-5e31a4a33399b-480x698.png
www.searchenginejournal.com/wp-content/uploads/2020/01/
68 KB
68 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/01/moz-robots-5e31a4a33399b-480x698.png
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
31a87e09bc3fc7ccb2149b418299ecae339b11000c5c0c4f0ff574cbe8b2318d

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Wed, 29 Jan 2020 15:39:04 GMT
server
nginx
etag
"5e31a718-11008"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
69640
x-request-id
d09faf5ec4c8bdb6d9a5200c6bf92b3a
expires
Thu, 31 Dec 2037 23:55:55 GMT
semrush-robots-5e31a53b7359b-480x810.png
www.searchenginejournal.com/wp-content/uploads/2020/01/
39 KB
39 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/01/semrush-robots-5e31a53b7359b-480x810.png
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
022c491ef985ac446182294ceb47e595ecdbb537300c51e2ec10fc47387c62f7

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Wed, 29 Jan 2020 15:38:31 GMT
server
nginx
etag
"5e31a6f7-9ac6"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
39622
x-request-id
d09faf5ec4c8bdb6d9a5200c6bf92b3a
expires
Thu, 31 Dec 2037 23:55:55 GMT
webceo-robots-5e31b3b1209b8-480x200.png
www.searchenginejournal.com/wp-content/uploads/2020/01/
15 KB
15 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/01/webceo-robots-5e31b3b1209b8-480x200.png
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
0247df6b9220981fc47473c27043ad09602f542a58d563f44ae5a4e91b827329

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Wed, 29 Jan 2020 16:37:35 GMT
server
nginx
etag
"5e31b4cf-3c74"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
15476
x-request-id
d09faf5ec4c8bdb6d9a5200c6bf92b3a
expires
Thu, 31 Dec 2037 23:55:55 GMT
webceo-robots-5e31b3b1209b8.png
www.searchenginejournal.com/wp-content/uploads/2020/01/
13 KB
13 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/01/webceo-robots-5e31b3b1209b8.png
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
28fb91355106692115c5ca2c04cd15b441692c106ed07bd4cd398d14568b8f0f

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Wed, 29 Jan 2020 16:37:35 GMT
server
nginx
etag
"5e31b4cf-340c"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
13324
x-request-id
d09faf5ec4c8bdb6d9a5200c6bf92b3a
expires
Thu, 31 Dec 2037 23:55:55 GMT
mcays-robots-5e31b3c91d769.png
www.searchenginejournal.com/wp-content/uploads/2020/01/
20 KB
20 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/01/mcays-robots-5e31b3c91d769.png
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
d888d58ad34eaf2c795253ddc129cd07e5d7904bf6583614a4338e21901cd4c0

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Wed, 29 Jan 2020 16:37:12 GMT
server
nginx
etag
"5e31b4b8-5054"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
20564
x-request-id
d09faf5ec4c8bdb6d9a5200c6bf92b3a
expires
Thu, 31 Dec 2037 23:55:55 GMT
leadfeeder-sitemap-5e31b5248237b.png
www.searchenginejournal.com/wp-content/uploads/2020/01/
7 KB
7 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/01/leadfeeder-sitemap-5e31b5248237b.png
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
abd2c31c501f8efcd7dcb8bfda286fd337d79183b340d7fd1bc81a37f8d8a1cc

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Wed, 29 Jan 2020 16:43:52 GMT
server
nginx
etag
"5e31b648-1c4a"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
7242
x-request-id
d09faf5ec4c8bdb6d9a5200c6bf92b3a
expires
Thu, 31 Dec 2037 23:55:55 GMT
robots-no-sitemap-5e31b57fda2d3.png
www.searchenginejournal.com/wp-content/uploads/2020/01/
3 KB
3 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/01/robots-no-sitemap-5e31b57fda2d3.png
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
d63969cac377eba8dc163b0ba842d53d263a6bbb31f1a6e59f3314da8d02de11

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Wed, 29 Jan 2020 16:43:31 GMT
server
nginx
etag
"5e31b633-b66"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
2918
x-request-id
d09faf5ec4c8bdb6d9a5200c6bf92b3a
expires
Thu, 31 Dec 2037 23:55:55 GMT
robots-correct-sitemap-5e31b5920f8b4-480x126.png
www.searchenginejournal.com/wp-content/uploads/2020/01/
7 KB
7 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/01/robots-correct-sitemap-5e31b5920f8b4-480x126.png
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
901a0b166ebe1970fd7e9d6d1c85791718ab530f1c63d293e9192063cebd20bf

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Wed, 29 Jan 2020 16:41:20 GMT
server
nginx
etag
"5e31b5b0-1bc6"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
7110
x-request-id
d09faf5ec4c8bdb6d9a5200c6bf92b3a
expires
Thu, 31 Dec 2037 23:55:55 GMT
noindex-thank-you-5e31b5aaed1c0.png
www.searchenginejournal.com/wp-content/uploads/2020/01/
6 KB
7 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/01/noindex-thank-you-5e31b5aaed1c0.png
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
cf79a232f8fe7bd21ac01c89f6e1c3774e8914bf8ae900ae979877504662699d

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Wed, 29 Jan 2020 16:41:36 GMT
server
nginx
etag
"5e31b5c0-19a0"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
6560
x-request-id
d09faf5ec4c8bdb6d9a5200c6bf92b3a
expires
Thu, 31 Dec 2037 23:55:55 GMT
robots-thank-you-5e31b5d3c39b6.png
www.searchenginejournal.com/wp-content/uploads/2020/01/
5 KB
5 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/01/robots-thank-you-5e31b5d3c39b6.png
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
61fe6af974028ea3358d36fa08e7b5a7df61010e4e91f2ff33914012a24084fe

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Wed, 29 Jan 2020 16:42:37 GMT
server
nginx
etag
"5e31b5fd-1460"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
5216
x-request-id
d09faf5ec4c8bdb6d9a5200c6bf92b3a
expires
Thu, 31 Dec 2037 23:55:55 GMT
robots-google-search-console-5e31b5f56f430.png
www.searchenginejournal.com/wp-content/uploads/2020/01/
21 KB
22 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/01/robots-google-search-console-5e31b5f56f430.png
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
34cf4cbc97aba634d40287ae9d58896d84105879cd2173ad0361662222f94aee

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Wed, 29 Jan 2020 16:43:28 GMT
server
nginx
etag
"5e31b630-55e8"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
21992
x-request-id
d09faf5ec4c8bdb6d9a5200c6bf92b3a
expires
Thu, 31 Dec 2037 23:55:55 GMT
leadfeeder-thank-you-page-5e31b63bebd87.png
www.searchenginejournal.com/wp-content/uploads/2020/01/
397 KB
398 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/01/leadfeeder-thank-you-page-5e31b63bebd87.png
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
62901637b079ad004302620acc29667e05931a0bf2f8f8f0a992f863b7a0febd

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Wed, 29 Jan 2020 16:44:40 GMT
server
nginx
etag
"5e31b678-6343a"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
406586
x-request-id
d09faf5ec4c8bdb6d9a5200c6bf92b3a
expires
Thu, 31 Dec 2037 23:55:55 GMT
noindex-nofollow-robots-5e31b6685081d.png
www.searchenginejournal.com/wp-content/uploads/2020/01/
371 KB
371 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2020/01/noindex-nofollow-robots-5e31b6685081d.png
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
3da9f17f868d54688d41272be758db97faba155afdf559ade101edd299fb2b62

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Wed, 29 Jan 2020 16:50:34 GMT
server
nginx
etag
"5e31b7da-5cada"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
379610
x-request-id
d09faf5ec4c8bdb6d9a5200c6bf92b3a
expires
Thu, 31 Dec 2037 23:55:55 GMT
truncated
/
358 B
0
Image
General
Full URL
data:truncated
Protocol
DATA
Server
-, , ASN (),
Reverse DNS
Software
/
Resource Hash
bf0808bd9c80866718bc07d9445c6fdadadabe21cc341d4fe156a1b09708d3fd

Request headers

accept-language
en-GB,en;q=0.9
Referer
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

Content-Type
image/svg+xml
state
api.btloader.com/mw/
0
101 B
Fetch
General
Full URL
https://api.btloader.com/mw/state?bt_env=prod
Requested by
Host: btloader.com
URL: https://btloader.com/tag?o=5132149699641344&upapi=true
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
130.211.23.194 Kansas City, United States, ASN396982 (GOOGLE-CLOUD-PLATFORM, US),
Reverse DNS
194.23.211.130.bc.googleusercontent.com
Software
/
Resource Hash
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

access-control-allow-origin
*
date
Fri, 15 Dec 2023 14:48:18 GMT
via
1.1 google
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
vary
Origin
px.gif
ad-delivery.net/
43 B
916 B
Image
General
Full URL
https://ad-delivery.net/px.gif?ch=2
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2606:4700:20::681a:246 , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
cf4724b2f736ed1a0ae6bc28f1ead963d9cd2c1fd87b6ef32e7799fc1c5c8bda

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:18 GMT
cf-cache-status
HIT
nel
{"success_fraction":0,"report_to":"cf-nel","max_age":604800}
age
1405960
x-guploader-uploadid
ABPtcPr3Z3wv0SXTY5ZkP0qwbllEhCNOVFWSMvJ-19BFEq6FmqTXHyZx0-K-bWot0jS8HjU78jY
x-goog-storage-class
MULTI_REGIONAL
x-goog-metageneration
5
x-goog-stored-content-encoding
identity
content-length
43
last-modified
Wed, 05 May 2021 19:25:32 GMT
server
cloudflare
etag
"ad4b0f606e0f8465bc4c4c170b37e1a3"
vary
Accept-Encoding
x-goog-generation
1620242732037093
content-type
image/gif
access-control-allow-origin
*
x-goog-hash
crc32c=cpEfJQ==, md5=rUsPYG4PhGW8TEwXCzfhow==
access-control-expose-headers
*, Content-Length, Date, Server, Transfer-Encoding, X-GUploader-UploadID, X-Google-Trace
cache-control
public, max-age=86400
report-to
{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=jQBFDOB3QMj6E6eEPPPgpDMdb10RrHwf%2BjGkQXD%2B3eakiPB9oA%2FrJ7gtocfD6oXCNoeuPrC9bdiRnoWxqshFlXeOB1tXp6P1yMKl%2FTnNqxupViO5uLuxTHuI5M7toM90lkoiwb58%2FNJls2DBQw%3D%3D"}],"group":"cf-nel","max_age":604800}
x-goog-stored-content-length
43
accept-ranges
bytes
cf-ray
835f7c98ecf22406-LHR
expires
Wed, 29 Nov 2023 08:19:21 GMT
favicon.ico
ad.doubleclick.net/
1 KB
571 B
Image
General
Full URL
https://ad.doubleclick.net/favicon.ico?ad=300x250&ad_box_=1&adnet=1&showad=1&size=250x250
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
142.250.181.230 , United States, ASN15169 (GOOGLE, US),
Reverse DNS
fra16s56-in-f6.1e100.net
Software
sffe /
Resource Hash
d961b08e4321250926de6f79087594975fe20ad1518de8f91eb711af5d1a6ef8
Security Headers
Name Value
X-Content-Type-Options nosniff
X-Xss-Protection 0

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 04:01:09 GMT
content-encoding
gzip
x-content-type-options
nosniff
age
38829
cross-origin-resource-policy
cross-origin
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length
104
x-xss-protection
0
last-modified
Tue, 08 May 2012 13:08:06 GMT
server
sffe
vary
Accept-Encoding
report-to
{"group":"ads-doubleclick-media","max_age":2592000,"endpoints":[{"url":"https://csp.withgoogle.com/csp/report-to/ads-doubleclick-media"}]}
content-type
image/x-icon
access-control-allow-origin
*
cache-control
public, max-age=86400
accept-ranges
bytes
cross-origin-opener-policy-report-only
same-origin; report-to="ads-doubleclick-media"
expires
Sat, 16 Dec 2023 04:01:09 GMT
px.gif
ad-delivery.net/
43 B
339 B
Image
General
Full URL
https://ad-delivery.net/px.gif?ch=1&e=0.3038178433602945
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2606:4700:20::681a:246 , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
cf4724b2f736ed1a0ae6bc28f1ead963d9cd2c1fd87b6ef32e7799fc1c5c8bda

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:18 GMT
cf-cache-status
HIT
nel
{"success_fraction":0,"report_to":"cf-nel","max_age":604800}
age
1405960
x-guploader-uploadid
ABPtcPr3Z3wv0SXTY5ZkP0qwbllEhCNOVFWSMvJ-19BFEq6FmqTXHyZx0-K-bWot0jS8HjU78jY
x-goog-storage-class
MULTI_REGIONAL
x-goog-metageneration
5
x-goog-stored-content-encoding
identity
content-length
43
last-modified
Wed, 05 May 2021 19:25:32 GMT
server
cloudflare
etag
"ad4b0f606e0f8465bc4c4c170b37e1a3"
vary
Accept-Encoding
x-goog-generation
1620242732037093
content-type
image/gif
access-control-allow-origin
*
x-goog-hash
crc32c=cpEfJQ==, md5=rUsPYG4PhGW8TEwXCzfhow==
access-control-expose-headers
*, Content-Length, Date, Server, Transfer-Encoding, X-GUploader-UploadID, X-Google-Trace
cache-control
public, max-age=86400
report-to
{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=mESxpbUxXudnOjEjTCvvafHe2JGULBFofLAQtFr4nCp7vHdMPD%2FQtKMkYEwZ2Dul0N8PlhloUI6eqTT93%2F59%2B0lRR6mEn5zZ6KMar09%2BbCePQfcbCjbmYQ2PR0oCAoW07jG5nMnUUHHk14XAEg%3D%3D"}],"group":"cf-nel","max_age":604800}
x-goog-stored-content-length
43
accept-ranges
bytes
cf-ray
835f7c98ecf62406-LHR
expires
Wed, 29 Nov 2023 08:19:21 GMT
sidebar1x-6579cf54d250a-sej.png
www.searchenginejournal.com/wp-content/uploads/2023/12/
28 KB
28 KB
Image
General
Full URL
https://www.searchenginejournal.com/wp-content/uploads/2023/12/sidebar1x-6579cf54d250a-sej.png
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
76895c9b99fdb460c5ad04c00aa06c2d4e47aab67a8b98520f2adf20dacde846

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
last-modified
Wed, 13 Dec 2023 15:35:50 GMT
server
nginx
etag
"6579cf56-6fde"
x-presslabs-cache
HIT
vary
Accept, Accept-Encoding
content-type
image/webp
cache-control
max-age = 315360000
accept-ranges
bytes
content-length
28638
x-request-id
a2828ce840e962efe31f0c059688711a
expires
Thu, 31 Dec 2037 23:55:55 GMT
/
cmp.uniconsent.com/json/
67 B
537 B
Fetch
General
Full URL
https://cmp.uniconsent.com/json/
Requested by
Host: cmp.uniconsent.com
URL: https://cmp.uniconsent.com/v2/main-v3.min.js
Protocol
H3
Security
QUIC, , AES_128_GCM
Server
2606:4700:3031::ac43:d3be , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
abbfa9c5b2e5ccdcd77eed22c4862ddf1b0d83fde488da934fe67153780ba212

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:17 GMT
content-encoding
br
cf-cache-status
DYNAMIC
nel
{"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server
cloudflare
vary
Origin
report-to
{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=c%2FbjFgp3OiH5qfUN3gDxTu3L2dWFa4uuV7VfKGbDp%2FjETR47bBjI9bOvZrvrqBrnyZPfcvnNZ%2F8bnRuWzvzravWz%2BNkqpovCMR2lh0C5znPVB4MKEaOWw6O%2Biu7JCpdZT1AGvQmZBVtKkQm%2FfQIRZpA%3D"}],"group":"cf-nel","max_age":604800}
content-type
application/json
access-control-allow-origin
*
cache-control
no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0
access-control-allow-credentials
true
cf-ray
835f7c97dda563f9-LHR
alt-svc
h3=":443"; ma=86400
vendor-list.json
cmp.uniconsent.com/v3/
558 KB
70 KB
Fetch
General
Full URL
https://cmp.uniconsent.com/v3/vendor-list.json
Requested by
Host: cmp.uniconsent.com
URL: https://cmp.uniconsent.com/v2/main-v3.min.js
Protocol
H3
Security
QUIC, , AES_128_GCM
Server
2606:4700:3031::ac43:d3be , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
59c66051ef0f633b25e7cdf631bbd1c9289a000e55f18bac9373f231c9bdaa5f

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:18 GMT
content-encoding
br
cf-cache-status
DYNAMIC
last-modified
Thu, 14 Dec 2023 19:10:02 GMT
nel
{"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server
cloudflare
etag
W/"9c862ab94bbad38491602820eaf8ccc5"
x-cache-status
HIT
report-to
{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=SXxbhFLoQbNGyZXpPWV6TfJyyLiDe1X9YgTi4S0fjWQqSHcqMy%2FtK7zrxLNw0x60ZDMSRxGabSM0%2BZOyXMppYCdPgTNWdO0wpHnS4fgiSPcpjgQYLV75r7o1eKfGtHZk%2FpQzNovB%2B9f7by2ukuVQUA0%3D"}],"group":"cf-nel","max_age":604800}
content-type
application/json
access-control-allow-origin
*
cache-control
max-age=86400
cf-ray
835f7c99180163f9-LHR
alt-svc
h3=":443"; ma=86400
4d881e84-98c1-4a13-ba07-b35ad4bb4dbc
config.aps.amazon-adsystem.com/configs/
564 B
839 B
Script
General
Full URL
https://config.aps.amazon-adsystem.com/configs/4d881e84-98c1-4a13-ba07-b35ad4bb4dbc
Requested by
Host: c.amazon-adsystem.com
URL: https://c.amazon-adsystem.com/aax2/apstag.js
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
99.86.4.128 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS
server-99-86-4-128.fra6.r.cloudfront.net
Software
CloudFront /
Resource Hash
6e122ad3e684634a13a0c1cca44c145bfedd3ed864d06fae1d2f2253a1d035e2

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:35:17 GMT
via
1.1 163be08bc1bc44818353c4fd88655bee.cloudfront.net (CloudFront)
server
CloudFront
x-amz-cf-pop
FRA6-C1
age
781
x-cache
Hit from cloudfront
content-type
application/javascript
cache-control
max-age=3600
content-length
564
x-amz-cf-id
VlliPnwART7dGnzHv3AJ3_WbKFV6uSlYCmQdPBlUshtimxNLwYX85g==
config
c.amazon-adsystem.com/cdn/prod/
3 KB
4 KB
XHR
General
Full URL
https://c.amazon-adsystem.com/cdn/prod/config?src=600&u=https%3A%2F%2Fwww.searchenginejournal.com&pubid=4d881e84-98c1-4a13-ba07-b35ad4bb4dbc
Requested by
Host: c.amazon-adsystem.com
URL: https://c.amazon-adsystem.com/aax2/apstag.js
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
99.84.149.113 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS
server-99-84-149-113.txl52.r.cloudfront.net
Software
Server /
Resource Hash
4feba251a69abc0926f05804e759752ad346892f397ffdac23ab18a7b5b009e8

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:30:52 GMT
via
1.1 f66aaf2d9f6a8e65595a1e24d5f18bcc.cloudfront.net (CloudFront)
server
Server
x-amz-cf-pop
TXL52-C1
age
1045
x-cache
Hit from cloudfront
content-type
application/json;charset=UTF-8
access-control-allow-origin
https://www.searchenginejournal.com
cache-control
max-age=21550, s-maxage=21600
access-control-allow-credentials
true
content-length
3337
x-amz-cf-id
79hVuCM8qMGlek-tyD1kU49tgpm5e62A5gkjLri4WY-0h6HQmI9L6g==
aps_csm.js
c.amazon-adsystem.com/bao-csm/aps-comm/
6 KB
3 KB
XHR
General
Full URL
https://c.amazon-adsystem.com/bao-csm/aps-comm/aps_csm.js
Requested by
Host: c.amazon-adsystem.com
URL: https://c.amazon-adsystem.com/aax2/apstag.js
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
99.84.149.113 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS
server-99-84-149-113.txl52.r.cloudfront.net
Software
AmazonS3 /
Resource Hash
06b99248a163333e36980a6cfb756f1a7de60fa49517162b87b1a44d5d48f844

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

x-amz-version-id
9yABOonr2HqHtwbarUcdbIqN0f4A8Qog
content-encoding
gzip
via
1.1 df2b5b3f847bbe9ad7f475c0831bcc3a.cloudfront.net (CloudFront)
date
Fri, 15 Dec 2023 08:23:47 GMT
x-amz-cf-pop
TXL52-C1
age
32709
x-amz-server-side-encryption
AES256
x-cache
Hit from cloudfront
last-modified
Tue, 29 Aug 2023 08:30:37 GMT
server
AmazonS3
etag
W/"a4d296427fc806b21335359e398c025c"
access-control-max-age
3000
access-control-allow-methods
GET
content-type
application/javascript
access-control-allow-origin
*
cache-control
public, max-age=86400
vary
Accept-Encoding,Origin
x-amz-cf-id
8UGLEOTUjAebxmh_NEtp1O-MCH4ziW2pS4gnADTKrGCev9AU6DN4kQ==
pubads_impl.js
securepubads.g.doubleclick.net/pagead/managed/js/gpt/m202312070101/
432 KB
135 KB
Script
General
Full URL
https://securepubads.g.doubleclick.net/pagead/managed/js/gpt/m202312070101/pubads_impl.js?cb=31080057
Requested by
Host: securepubads.g.doubleclick.net
URL: https://securepubads.g.doubleclick.net/tag/js/gpt.js
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2a00:1450:4001:829::2002 Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),
Reverse DNS
Software
cafe /
Resource Hash
013fc39efb38a28d8eccab58189059646847bc5c54e1c4b637e874b6109ee0ef
Security Headers
Name Value
X-Content-Type-Options nosniff
X-Xss-Protection 0

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 08:52:45 GMT
content-encoding
br
x-content-type-options
nosniff
age
21333
p3p
policyref="https://googleads.g.doubleclick.net/pagead/gcn_p3p_.xml", CP="CURa ADMa DEVa TAIo PSAo PSDo OUR IND UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR"
cross-origin-resource-policy
cross-origin
content-disposition
attachment; filename="f.txt"
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length
138203
x-xss-protection
0
server
cafe
etag
14959461090202361603
vary
Accept-Encoding
content-type
text/javascript; charset=UTF-8
cache-control
public, immutable, max-age=31536000
timing-allow-origin
*
expires
Sat, 14 Dec 2024 08:52:45 GMT
country
api.btloader.com/
16 B
132 B
Fetch
General
Full URL
https://api.btloader.com/country
Requested by
Host: btloader.com
URL: https://btloader.com/tag?o=5132149699641344&upapi=true
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
130.211.23.194 Kansas City, United States, ASN396982 (GOOGLE-CLOUD-PLATFORM, US),
Reverse DNS
194.23.211.130.bc.googleusercontent.com
Software
/
Resource Hash
4304ad9301017a03d77d588218c73d2b5fbc0a292138994c023a10aed3196740

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:18 GMT
via
1.1 google
vary
Origin
content-type
application/json
access-control-allow-origin
*
cache-control
private, max-age=300, stale-while-revalidate=600, stale-if-error=600
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length
16
pv
api.btloader.com/
0
66 B
XHR
General
Full URL
https://api.btloader.com/pv?tid=HMcC6g3n&w=5082459048247296&o=5132149699641344&cv=2.1.26&widget=false&r=false&vr=1600x1200&pageURL=https%3A%2F%2Fwww.searchenginejournal.com%2Ftechnical-seo%2Fmeta-robots-tags-robots-txt%2F&sid=pLmOIwgJ&pm=true&upapi=true
Requested by
Host: btloader.com
URL: https://btloader.com/tag?o=5132149699641344&upapi=true
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
130.211.23.194 Kansas City, United States, ASN396982 (GOOGLE-CLOUD-PLATFORM, US),
Reverse DNS
194.23.211.130.bc.googleusercontent.com
Software
/
Resource Hash
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

access-control-allow-origin
*
date
Fri, 15 Dec 2023 14:48:18 GMT
cache-control
no-cache, no-store, must-revalidate
via
1.1 google
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
vary
Origin
ats.js
ats.rlcdn.com/
110 KB
35 KB
Script
General
Full URL
https://ats.rlcdn.com/ats.js
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
13.227.219.68 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS
server-13-227-219-68.ams54.r.cloudfront.net
Software
AmazonS3 /
Resource Hash
a2aa2577c105dab138246b4e0a1f575b3c92c30d5aced108d3f73897bd46823f

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

x-amz-version-id
x5VtTe.o38AhKNl9GXJ.IeIaII4uy0GZ
content-encoding
br
via
1.1 80826ca6c4fd6005aeacf5a03c8d42e8.cloudfront.net (CloudFront)
date
Fri, 15 Dec 2023 12:27:18 GMT
last-modified
Thu, 19 Oct 2023 08:25:12 GMT
server
AmazonS3
x-amz-cf-pop
AMS54-C1
age
8461
x-amz-server-side-encryption
AES256
etag
W/"b248cc9d0fdeb36bdeb7efabad1132ee"
vary
Accept-Encoding
x-cache
Hit from cloudfront
content-type
text/javascript
x-amz-cf-id
VzLaKezWXFJEGREbvy_PC--s8XpWNloMQMC03b0iqpd3mcejQw3PyQ==
pubcid.min.js
secure.cdn.fastclick.net/js/pubcid/latest/
54 KB
17 KB
Script
General
Full URL
https://secure.cdn.fastclick.net/js/pubcid/latest/pubcid.min.js
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_256_GCM
Server
184.30.211.26 Frankfurt am Main, Germany, ASN16625 (AKAMAI-AS, US),
Reverse DNS
a184-30-211-26.deploy.static.akamaitechnologies.com
Software
Apache /
Resource Hash
43f804d38a294c6df1ce8ee64fb95ad0ff5a8d6d5685d9537df02212668a1dff

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:18 GMT
content-encoding
gzip
last-modified
Mon, 23 Jan 2023 19:40:17 GMT
server
Apache
etag
"d734-5f2f3919e751f-gzip"
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age=900
accept-ranges
bytes
content-length
17407
expires
Fri, 15 Dec 2023 15:03:18 GMT
sync.min.js
tags.crwdcntrl.net/lt/c/16576/
39 KB
12 KB
Script
General
Full URL
https://tags.crwdcntrl.net/lt/c/16576/sync.min.js
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
18.64.79.46 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS
server-18-64-79-46.txl50.r.cloudfront.net
Software
AmazonS3 /
Resource Hash
7713183ba1a38b1ea2be2d5f7d3d49dab7b8d468cf78a603e6517ffbd1f33d59

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 01:52:33 GMT
content-encoding
gzip
via
1.1 6ffd9144e1bc0d7518dfec3f04843cdc.cloudfront.net (CloudFront)
last-modified
Wed, 06 Sep 2023 15:56:42 GMT
server
AmazonS3
x-amz-cf-pop
TXL50-P2
age
46546
x-amz-server-side-encryption
AES256
etag
W/"6e8b1f94eaf615b7d0953ad4e8d8bb85"
vary
Accept-Encoding
x-cache
Hit from cloudfront
content-type
text/javascript
cache-control
public, max-age=86400
x-amz-cf-id
wuG2hnf3ji1wiN1_Llsz0NbVcSdM_dkWlPLFzGbwqsi5-Nq2d-DoRQ==
hadron.js
cdn.hadronid.net/
55 KB
10 KB
Script
General
Full URL
https://cdn.hadronid.net/hadron.js?url=https%3A%2F%2Fwww.searchenginejournal.com%2Ftechnical-seo%2Fmeta-robots-tags-robots-txt%2F&ref=&_it=amazon&partner_id=437
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2606:4700:10::ac43:246e , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
2365cc11ef3d43f265b848c7164e5487c7a49d6af06c2938ac9272c8d91fc1a2

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:18 GMT
content-encoding
gzip
cf-cache-status
HIT
cf-bgj
minify
last-modified
Wed, 29 Nov 2023 15:31:45 GMT
server
cloudflare
x-amz-request-id
01C975WVAA3JDKHJ
age
6841
etag
W/"13043c1bbaf21ccc6e8ed474a744d3f2"
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age=3600
cf-ray
835f7c9a58ad24b5-LHR
x-amz-id-2
hxb2Br8vf2uBBniSeqakgjztWKIPtATtD6bN1gV+n3Vg6mNMl2rDRQL+gF8pDyHWIII6UuGjSsI=
id5-api.js
cdn.id5-sync.com/api/1.0/
151 KB
33 KB
Script
General
Full URL
https://cdn.id5-sync.com/api/1.0/id5-api.js
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2606:4700:10::6816:3556 , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
12ba93db33de679d443dc28aee4a2190b580b8ad3fc53216d5bb2678d4e17f29
Security Headers
Name Value
Strict-Transport-Security max-age=15552000; includeSubDomains; preload

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:18 GMT
strict-transport-security
max-age=15552000; includeSubDomains; preload
content-encoding
gzip
cf-cache-status
HIT
last-modified
Thu, 07 Dec 2023 12:57:20 GMT
server
cloudflare
x-amz-request-id
AQH5J6GNGTMB7RN3
age
843
etag
W/"7229163a9092e2cee472ddee92dcb6ba"
x-amz-server-side-encryption
AES256
vary
Accept-Encoding
content-type
text/javascript;charset=utf-8
cache-control
public, max-age=3600
cf-ray
835f7c9a4e064183-LHR
x-amz-id-2
4rQGyDOq0TlopguYPKLJljwt+nVsEDJSJzJemNgKAQZSDW7J3Myyj+bS0MjmO5X6cHvVzIfsqbA=
additional-consent-providers.json
cmp.uniconsent.com/v2/7d36e67838/
55 KB
15 KB
Fetch
General
Full URL
https://cmp.uniconsent.com/v2/7d36e67838/additional-consent-providers.json
Requested by
Host: cmp.uniconsent.com
URL: https://cmp.uniconsent.com/v2/main-v3.min.js
Protocol
H3
Security
QUIC, , AES_128_GCM
Server
2606:4700:3031::ac43:d3be , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
18c4f9d98f61be71d13dbb438be923a05012c58390c394919a98c6ef7308d93c

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:18 GMT
content-encoding
br
cf-cache-status
DYNAMIC
last-modified
Sun, 12 Nov 2023 00:28:55 GMT
nel
{"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server
cloudflare
etag
W/"cfdbe06d439d84892aa1176a959c4706"
x-cache-status
HIT
report-to
{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=8iWcMaNAAqXfYMd3H3en6yjPVeBQYK05cBeMUi3JCOKTZlzvGzaUqtnYjw4EopaFaKfvSpkRZuRvE11cORn%2FyCdF%2Fx2WPFHNHP%2FWLH6pkuYcgC5Rsm4VNkaW9Vk8j2nXXk%2FiPb1bCOBrMSq4cFX01Vo%3D"}],"group":"cf-nel","max_age":604800}
content-type
application/json
access-control-allow-origin
*
cache-control
max-age=86400
cf-ray
835f7c99f94f63f9-LHR
alt-svc
h3=":443"; ma=86400
vendors-v3.js
cmp.uniconsent.com/v2/
109 KB
13 KB
Script
General
Full URL
https://cmp.uniconsent.com/v2/vendors-v3.js
Requested by
Host: cmp.uniconsent.com
URL: https://cmp.uniconsent.com/v2/main-v3.min.js
Protocol
H3
Security
QUIC, , AES_128_GCM
Server
2606:4700:3031::ac43:d3be , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
db3a067eb9d66b2baef7f7019877cae166bfd30d4cf801a244d1cc8ade33e61e

Request headers

Referer
https://www.searchenginejournal.com/
Origin
https://www.searchenginejournal.com
accept-language
en-GB,en;q=0.9
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:18 GMT
content-encoding
br
cf-cache-status
HIT
nel
{"success_fraction":0,"report_to":"cf-nel","max_age":604800}
age
57285
x-cache-status
HIT
alt-svc
h3=":443"; ma=86400
last-modified
Wed, 13 Dec 2023 22:49:26 GMT
server
cloudflare
etag
W/"dae2aac8a4256f9d84ce4bde415fd751"
vary
Accept-Encoding
report-to
{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=CvongJPtKMRWGE3zJtPE777%2B0%2FWDZMrCmwalyLozesQl4WMotcCJgAuTkzmpgAluI23ARJ0twYq5G6gkP1TplI41Tk9qO6LAt48HOZtLonOAz%2FQoc4R902Bl8hFiGNlk%2FDXXvSMZV9bRmiMKEDkSjbo%3D"}],"group":"cf-nel","max_age":604800}
content-type
application/javascript; charset=utf-8
access-control-allow-origin
*
cache-control
max-age=86400
cf-ray
835f7c9aca5d63f9-LHR
hadron.json
id.hadron.ad.gt/v1/
108 B
298 B
XHR
General
Full URL
https://id.hadron.ad.gt/v1/hadron.json?_it=amazon&partner_id=437&sync=0&domain=www.searchenginejournal.com&url=https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Requested by
Host: cdn.hadronid.net
URL: https://cdn.hadronid.net/hadron.js?url=https%3A%2F%2Fwww.searchenginejournal.com%2Ftechnical-seo%2Fmeta-robots-tags-robots-txt%2F&ref=&_it=amazon&partner_id=437
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2606:4700:10::6816:445 , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
865519f76b49f44e52ea431a295b39c46cdb31ce47822104fad3c4e2e80cb4e7

Request headers

Referer
https://www.searchenginejournal.com/
accept-language
en-GB,en;q=0.9
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36
Content-Type
application/json

Response headers

date
Fri, 15 Dec 2023 14:48:18 GMT
content-encoding
gzip
cf-cache-status
DYNAMIC
server
cloudflare
access-control-allow-methods
HEAD,GET,POST,PUT,DELETE,OPTIONS
content-type
application/json
access-control-allow-origin
*
cache-control
private,max-age=30
access-control-allow-credentials
true
debug
NON-OPTIONS
access-control-allow-headers
authorization
cf-ray
835f7c9c185148c9-LHR
hadron.json
id.hadron.ad.gt/v1/
0
0
Preflight
General
Full URL
https://id.hadron.ad.gt/v1/hadron.json?_it=amazon&partner_id=437&sync=0&domain=www.searchenginejournal.com&url=https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2606:4700:10::6816:445 , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash

Request headers

Accept
*/*
Access-Control-Request-Headers
content-type
Access-Control-Request-Method
GET
Origin
https://www.searchenginejournal.com
Sec-Fetch-Mode
cors
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

access-control-allow-headers
Origin, X-Requested-With, Content-Type, Accept
access-control-allow-origin
*
allow
POST, OPTIONS, GET
cache-control
max-age=31536000 public, no-transform
cf-cache-status
DYNAMIC
cf-ray
835f7c9b6f7148c9-LHR
content-length
0
content-type
application/json
date
Fri, 15 Dec 2023 14:48:18 GMT
debug
OPTIONS block
expires
Sat, 14 Dec 2024 14:48:18 GMT
server
cloudflare
/
geo.privacymanager.io/
31 B
609 B
Fetch
General
Full URL
https://geo.privacymanager.io/
Requested by
Host: ats.rlcdn.com
URL: https://ats.rlcdn.com/ats.js
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
18.64.119.20 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS
server-18-64-119-20.txl50.r.cloudfront.net
Software
/
Resource Hash
8a78cef0b7848d146a9983a7ec0e37c321eef8a01d5847caa7483e5ecb86be04

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Thu, 14 Dec 2023 17:26:44 GMT
via
1.1 d9523e44e96d2539081596bb1d268d44.cloudfront.net (CloudFront), 1.1 bc66fd12bea603144bf0b6c1578cb3e0.cloudfront.net (CloudFront)
x-amz-cf-pop
FRA56-P3, TXL50-P4
age
76894
x-amzn-requestid
09f09985-1473-4ab6-925c-ec7da04f12fe
x-amzn-trace-id
Root=1-657b3ad4-73e9a331257bc8de5bf8d1d6;Sampled=0;lineage=06620786:0
access-control-allow-methods
DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT
content-type
application/json
access-control-allow-origin
*
x-cache
Hit from cloudfront
x-amz-apigw-id
P8YhNG2-DoEEYHw=
content-length
31
x-amz-cf-id
swsVAAO4fl5mRcKoBkd_dWW1JiA5QwMkcBNk1tY-Cwv5Qy7z1IQS1g==
access-control-allow-headers
Content-Type,Authorization,X-Amz-Date,X-Api-Key,X-Amz-Security-Token
map
bcp.crwdcntrl.net/6/
60 B
343 B
XHR
General
Full URL
https://bcp.crwdcntrl.net/6/map
Requested by
Host: tags.crwdcntrl.net
URL: https://tags.crwdcntrl.net/lt/c/16576/sync.min.js
Protocol
H2
Security
TLS 1.2, ECDHE_RSA, AES_128_GCM
Server
54.216.91.216 Dublin, Ireland, ASN16509 (AMAZON-02, US),
Reverse DNS
ec2-54-216-91-216.eu-west-1.compute.amazonaws.com
Software
Jetty(9.4.38.v20210224) /
Resource Hash
2289485ae05bbe8f0ec23f821aff7c01e1ebf720bb949d221790b68dba188408

Request headers

Referer
https://www.searchenginejournal.com/
accept-language
en-GB,en;q=0.9
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36
Content-Type
text/plain;charset=UTF-8

Response headers

pragma
no-cache
date
Fri, 15 Dec 2023 14:48:18 GMT
server
Jetty(9.4.38.v20210224)
content-type
application/json;charset=utf-8
p3p
CP=NOI DSP COR NID PSAa PSDa OUR UNI COM NAV
access-control-allow-origin
https://www.searchenginejournal.com
cache-control
no-cache
x-server
10.45.10.36
access-control-allow-credentials
true
content-length
60
expires
0
logo.svg
cmp.uniconsent.com/
11 KB
5 KB
Image
General
Full URL
https://cmp.uniconsent.com/logo.svg
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H3
Security
QUIC, , AES_128_GCM
Server
2606:4700:3031::ac43:d3be , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
6c76972839a5126e9e07e4958eb6b5ed3ec40985e62564c9d31adcac47486da6

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:18 GMT
content-encoding
br
cf-cache-status
HIT
nel
{"success_fraction":0,"report_to":"cf-nel","max_age":604800}
age
57467
x-cache-status
MISS
alt-svc
h3=":443"; ma=86400
last-modified
Thu, 18 May 2023 15:32:43 GMT
server
cloudflare
etag
W/"062b26e035a7c2a34f2118af82a83a7a"
vary
Accept-Encoding
report-to
{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=6iZ9piccWCywvWBdOB%2FgfSbfB7rzEoFNkPBDaB2H%2BIRcC%2FibP2JslnHduALmV1oS2U99a4CIbRSKFSutEac07MPI9oIEDIOdS4R9jyJBTVjNB8ElT2Fe1S%2FnD2kMiUDNhhqqvUG0zV9itsA0VNwJhy4%3D"}],"group":"cf-nel","max_age":604800}
content-type
image/svg+xml
access-control-allow-origin
*
cache-control
max-age=86400
cf-ray
835f7c9b48053853-LHR
v1
openfpcdn.io/botd/
15 KB
5 KB
Script
General
Full URL
https://openfpcdn.io/botd/v1
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
13.32.99.97 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS
server-13-32-99-97.fra60.r.cloudfront.net
Software
CloudFront /
Resource Hash
9ea87ca1966825417ae6feefde236700033bd308670bcc448d7094ac3cdacb7a
Security Headers
Name Value
Strict-Transport-Security max-age=63072000; includeSubDomains; preload
X-Content-Type-Options nosniff

Request headers

Referer
https://www.searchenginejournal.com/
Origin
https://www.searchenginejournal.com
accept-language
en-GB,en;q=0.9
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

strict-transport-security
max-age=63072000; includeSubDomains; preload
content-encoding
br
x-content-type-options
nosniff
date
Fri, 15 Dec 2023 14:17:31 GMT
via
1.1 7251dede1ac94066b27bcd33919b30c6.cloudfront.net (CloudFront)
x-amz-cf-pop
FRA60-P3
age
1850
x-cache
Hit from cloudfront
cross-origin-resource-policy
cross-origin
alt-svc
h3=":443"; ma=86400
server
CloudFront
etag
W/"2d6TtWT4658feqKfRmahAiSfkko"
vary
Accept-Encoding
content-type
text/javascript; charset=utf-8
access-control-allow-origin
*
cache-control
public, max-age=633873, s-maxage=11206
x-amz-cf-id
B83LEs1PFjx8Ov-6EAVfPfVplnVEshgjUct7ScxihutS-ZsGYeLwtQ==
437
a.ad.gt/api/v1/u/matches/
12 KB
4 KB
Script
General
Full URL
https://a.ad.gt/api/v1/u/matches/437?_it=amazon
Requested by
Host: cdn.hadronid.net
URL: https://cdn.hadronid.net/hadron.js?url=https%3A%2F%2Fwww.searchenginejournal.com%2Ftechnical-seo%2Fmeta-robots-tags-robots-txt%2F&ref=&_it=amazon&partner_id=437
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2606:4700:10::ac43:17ea , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
e3d695104174ae7c3034322cd7cd822375f51a1f5b0c121f3949ef38b6371353

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:18 GMT
content-encoding
gzip
cf-cache-status
HIT
last-modified
Fri, 15 Dec 2023 14:43:23 GMT
server
cloudflare
age
295
vary
Accept-Encoding
content-type
application/javascript
cache-control
max-age=14400
cross-origin-resource-policy
cross-origin
cf-ray
835f7c9d6c3248c3-LHR
gtm.js
www.googletagmanager.com/
276 KB
87 KB
Script
General
Full URL
https://www.googletagmanager.com/gtm.js?id=GTM-M2HM2DB
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2a00:1450:4001:80f::2008 Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),
Reverse DNS
Software
Google Tag Manager /
Resource Hash
292ab40cd9c46838d7c5450d5b159fbc62bf4d5663ae95512d599aaa9925ae89
Security Headers
Name Value
Strict-Transport-Security max-age=31536000; includeSubDomains
X-Xss-Protection 0

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:18 GMT
content-encoding
br
strict-transport-security
max-age=31536000; includeSubDomains
cross-origin-resource-policy
cross-origin
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length
88777
x-xss-protection
0
last-modified
Fri, 15 Dec 2023 12:00:00 GMT
server
Google Tag Manager
vary
Accept-Encoding
content-type
application/javascript; charset=UTF-8
access-control-allow-origin
*
cache-control
private, max-age=900
access-control-allow-credentials
true
access-control-allow-headers
Cache-Control
expires
Fri, 15 Dec 2023 14:48:18 GMT
js
www.googletagmanager.com/gtag/
299 KB
95 KB
Script
General
Full URL
https://www.googletagmanager.com/gtag/js?id=G-TJPZMKSHES&l=dataLayer&cx=c
Requested by
Host: www.googletagmanager.com
URL: https://www.googletagmanager.com/gtm.js?id=GTM-M2HM2DB
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2a00:1450:4001:80f::2008 Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),
Reverse DNS
Software
Google Tag Manager /
Resource Hash
30850680c0f24c91b3130c1563e324fd245ac50f21bd4d5cc7010429f606b744
Security Headers
Name Value
Strict-Transport-Security max-age=31536000; includeSubDomains
X-Xss-Protection 0

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:18 GMT
content-encoding
br
strict-transport-security
max-age=31536000; includeSubDomains
server
Google Tag Manager
vary
Accept-Encoding
content-type
application/javascript; charset=UTF-8
access-control-allow-origin
*
cache-control
private, max-age=900
access-control-allow-credentials
true
cross-origin-resource-policy
cross-origin
access-control-allow-headers
Cache-Control
content-length
96664
x-xss-protection
0
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
expires
Fri, 15 Dec 2023 14:48:18 GMT
js
www.googletagmanager.com/gtag/
292 KB
93 KB
Script
General
Full URL
https://www.googletagmanager.com/gtag/js?id=G-S08JX510ES&l=dataLayer&cx=c
Requested by
Host: www.googletagmanager.com
URL: https://www.googletagmanager.com/gtm.js?id=GTM-M2HM2DB
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2a00:1450:4001:80f::2008 Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),
Reverse DNS
Software
Google Tag Manager /
Resource Hash
4dc868e5e30c80252a63341c4a0b7b692f79503c78149d5deca3448210752421
Security Headers
Name Value
Strict-Transport-Security max-age=31536000; includeSubDomains
X-Xss-Protection 0

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:19 GMT
content-encoding
br
strict-transport-security
max-age=31536000; includeSubDomains
server
Google Tag Manager
vary
Accept-Encoding
content-type
application/javascript; charset=UTF-8
access-control-allow-origin
*
cache-control
private, max-age=900
access-control-allow-credentials
true
cross-origin-resource-policy
cross-origin
access-control-allow-headers
Cache-Control
content-length
94891
x-xss-protection
0
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
expires
Fri, 15 Dec 2023 14:48:19 GMT
js
www.googletagmanager.com/gtag/
282 KB
92 KB
Script
General
Full URL
https://www.googletagmanager.com/gtag/js?id=G-9QQPL212Z4&l=dataLayer&cx=c
Requested by
Host: www.googletagmanager.com
URL: https://www.googletagmanager.com/gtm.js?id=GTM-M2HM2DB
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2a00:1450:4001:80f::2008 Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),
Reverse DNS
Software
Google Tag Manager /
Resource Hash
253587dbc60fd8e8074a091f8e40f8259fe35d2a1a2e6671b845f1e4718fd548
Security Headers
Name Value
Strict-Transport-Security max-age=31536000; includeSubDomains
X-Xss-Protection 0

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:19 GMT
content-encoding
br
strict-transport-security
max-age=31536000; includeSubDomains
server
Google Tag Manager
vary
Accept-Encoding
content-type
application/javascript; charset=UTF-8
access-control-allow-origin
*
cache-control
private, max-age=900
access-control-allow-credentials
true
cross-origin-resource-policy
cross-origin
access-control-allow-headers
Cache-Control
content-length
94209
x-xss-protection
0
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
expires
Fri, 15 Dec 2023 14:48:19 GMT
lftracker_v1_ywVkO4XKPNd7Z6Bj.js
sc.lfeeder.com/
30 KB
11 KB
Script
General
Full URL
https://sc.lfeeder.com/lftracker_v1_ywVkO4XKPNd7Z6Bj.js
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2600:9000:2261:3200:4:d7e1:700:93a1 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS
Software
AmazonS3 /
Resource Hash
10dc140898ff0dbc97793f8f1f805d8f93b81cc0d80aa65d6b3040d4c71698fc

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

x-amz-version-id
WiB7wTU0ZMOhdRvIyDlnVDV5RalbDI1s
content-encoding
gzip
via
1.1 d58f8ae27ab00bbbe1dc072976204492.cloudfront.net (CloudFront)
date
Fri, 15 Dec 2023 14:48:19 GMT
last-modified
Thu, 05 Oct 2023 07:21:10 GMT
server
AmazonS3
x-amz-cf-pop
TXL50-P4
age
484
x-amz-server-side-encryption
AES256
etag
W/"bd5ed3833c2da121cb85b2541e4776d0"
vary
Accept-Encoding
x-cache
Hit from cloudfront
content-type
application/javascript
cache-control
max-age=3600
x-amz-cf-id
K09iSTnuIc0SOktXAMcOztcaT71lYshWMYurpafRKd7CEKP6MAIR5Q==
collect
region1.analytics.google.com/g/
0
263 B
Ping
General
Full URL
https://region1.analytics.google.com/g/collect?v=2&tid=G-TJPZMKSHES&gtm=45je3bt0v874297284z8836266955&_p=1702651698685&_gaz=1&gcd=11l1l1l1l1&dma=0&tcfd=10000&cid=1326972924.1702651699&ul=en-us&sr=1600x1200&ir=1&uaa=&uab=&uafvl=&uamb=0&uam=&uap=&uapv=&uaw=0&_eu=EA&_s=1&dl=https%3A%2F%2Fwww.searchenginejournal.com%2Ftechnical-seo%2Fmeta-robots-tags-robots-txt%2F&sid=1702651699&sct=1&seg=0&dt=Best%20Practices%20for%20Setting%20Up%20Meta%20Robots%20Tags%20%26%20Robots.txt&en=page_view&_fv=1&_nsi=1&_ss=1&ep.author_name=anna%20crowe&ep.publish_year=2020&ep.article_all_categories=seo&ep.article_word_count=2000%2B&ep.title_word_count=50%20-%2074&ep.publish_year_month=2020-04&ep.content_type=in-depth%20guides&ep.primary_categories=seo&ep.content_group=in-depth%20guides&ep.shares_count=more%20than%20500&ep.is_refresh=yes&ep.content_sections=seo&ep.infinte_scroll=no&ep.has_summary=no&ep.document_ref=&ep.pers_op=yes&ep.article_type=evergreen&tfd=2621
Requested by
Host: www.googletagmanager.com
URL: https://www.googletagmanager.com/gtag/js?id=G-TJPZMKSHES&l=dataLayer&cx=c
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2001:4860:4802:34::36 , United States, ASN15169 (GOOGLE, US),
Reverse DNS
Software
Golfe2 /
Resource Hash
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

pragma
no-cache
date
Fri, 15 Dec 2023 14:48:19 GMT
server
Golfe2
content-type
text/plain
access-control-allow-origin
https://www.searchenginejournal.com
cache-control
no-cache, no-store, must-revalidate
access-control-allow-credentials
true
cross-origin-resource-policy
cross-origin
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length
0
expires
Fri, 01 Jan 1990 00:00:00 GMT
collect
stats.g.doubleclick.net/g/
0
263 B
Ping
General
Full URL
https://stats.g.doubleclick.net/g/collect?v=2&tid=G-TJPZMKSHES&cid=1326972924.1702651699&gtm=45je3bt0v874297284z8836266955&aip=1&dma=0&gcd=11l1l1l1l1
Requested by
Host: www.googletagmanager.com
URL: https://www.googletagmanager.com/gtag/js?id=G-TJPZMKSHES&l=dataLayer&cx=c
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2a00:1450:400c:c00::9a Brussels, Belgium, ASN15169 (GOOGLE, US),
Reverse DNS
Software
Golfe2 /
Resource Hash
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

pragma
no-cache
date
Fri, 15 Dec 2023 14:48:19 GMT
server
Golfe2
content-type
text/plain
access-control-allow-origin
https://www.searchenginejournal.com
cache-control
no-cache, no-store, must-revalidate
access-control-allow-credentials
true
cross-origin-resource-policy
cross-origin
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length
0
expires
Fri, 01 Jan 1990 00:00:00 GMT
ga-audiences
www.google.co.uk/ads/
42 B
408 B
Image
General
Full URL
https://www.google.co.uk/ads/ga-audiences?v=1&t=sr&slf_rd=1&_r=4&tid=G-TJPZMKSHES&cid=1326972924.1702651699&gtm=45je3bt0v874297284z8836266955&aip=1&dma=0&gcd=11l1l1l1l1&z=691208003
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2a00:1450:4001:82a::2003 Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),
Reverse DNS
Software
cafe /
Resource Hash
ef1955ae757c8b966c83248350331bd3a30f658ced11f387f8ebf05ab3368629
Security Headers
Name Value
X-Content-Type-Options nosniff
X-Xss-Protection 0

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

pragma
no-cache
date
Fri, 15 Dec 2023 14:48:19 GMT
x-content-type-options
nosniff
server
cafe
content-type
image/gif
p3p
policyref="https://www.googleadservices.com/pagead/p3p.xml", CP="NOI DEV PSA PSD IVA IVD OTP OUR OTR IND OTC"
cache-control
no-cache, no-store, must-revalidate
cross-origin-resource-policy
cross-origin
timing-allow-origin
*
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length
42
x-xss-protection
0
expires
Fri, 01 Jan 1990 00:00:00 GMT
collect
region1.google-analytics.com/g/
0
254 B
Ping
General
Full URL
https://region1.google-analytics.com/g/collect?v=2&tid=G-S08JX510ES&gtm=45je3bt0v9103719431z8836266955&_p=1702651698685&gcd=11l1l1l1l1&dma=0&tcfd=10000&cid=1326972924.1702651699&ul=en-us&sr=1600x1200&uaa=&uab=&uafvl=&uamb=0&uam=&uap=&uapv=&uaw=0&_s=1&dl=https%3A%2F%2Fwww.searchenginejournal.com%2Ftechnical-seo%2Fmeta-robots-tags-robots-txt%2F&sid=1702651699&sct=1&seg=0&dt=Best%20Practices%20for%20Setting%20Up%20Meta%20Robots%20Tags%20%26%20Robots.txt&en=page_view&_fv=1&_ss=1&ep.author_name=anna%20crowe&ep.publish_year=2020&ep.article_all_categories=seo&ep.article_word_count=2000%2B&ep.title_word_count=50%20-%2074&ep.publish_year_month=2020-04&ep.content_type=in-depth%20guides&ep.primary_categories=seo&ep.content_group=seo&ep.shares_count=more%20than%20500&ep.is_refresh=yes&ep.content_sections=seo&ep.infinte_scroll=no&ep.has_summary=no&ep.document_ref=&ep.pers_op=yes&ep.article_type=evergreen&tfd=2655
Requested by
Host: www.googletagmanager.com
URL: https://www.googletagmanager.com/gtag/js?id=G-S08JX510ES&l=dataLayer&cx=c
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2001:4860:4802:32::36 , United States, ASN15169 (GOOGLE, US),
Reverse DNS
Software
Golfe2 /
Resource Hash
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

pragma
no-cache
date
Fri, 15 Dec 2023 14:48:19 GMT
server
Golfe2
content-type
text/plain
access-control-allow-origin
https://www.searchenginejournal.com
cache-control
no-cache, no-store, must-revalidate
access-control-allow-credentials
true
cross-origin-resource-policy
cross-origin
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length
0
expires
Fri, 01 Jan 1990 00:00:00 GMT
collect
region1.google-analytics.com/g/
0
45 B
Ping
General
Full URL
https://region1.google-analytics.com/g/collect?v=2&tid=G-9QQPL212Z4&gtm=45je3bt0v9121303883z8836266955&_p=1702651698685&gcd=11l1l1l1l1&dma=0&tcfd=10000&cid=1326972924.1702651699&ul=en-us&sr=1600x1200&uaa=&uab=&uafvl=&uamb=0&uam=&uap=&uapv=&uaw=0&_s=1&sid=1702651699&sct=1&seg=0&dl=https%3A%2F%2Fwww.searchenginejournal.com%2Ftechnical-seo%2Fmeta-robots-tags-robots-txt%2F&dt=Best%20Practices%20for%20Setting%20Up%20Meta%20Robots%20Tags%20%26%20Robots.txt&en=page_view&_fv=1&_ss=1&ep.author_name=anna%20crowe&ep.publish_year=2020&ep.article_all_categories=seo&ep.article_word_count=2000%2B&ep.title_word_count=50%20-%2074&ep.publish_year_month=2020-04&ep.content_type=in-depth%20guides&ep.primary_categories=seo&ep.content_group=in-depth%20guides&ep.shares_count=more%20than%20500&ep.is_refresh=yes&ep.page_url_no_params=www.searchenginejournal.com%2Ftechnical-seo%2Fmeta-robots-tags-robots-txt%2F&ep.content_sections=seo&ep.infinte_scroll=no&ep.has_summary=no&ep.document_ref=&ep.custom_geo_location=GB&ep.article_type=evergreen&tfd=2676
Requested by
Host: www.googletagmanager.com
URL: https://www.googletagmanager.com/gtag/js?id=G-9QQPL212Z4&l=dataLayer&cx=c
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2001:4860:4802:32::36 , United States, ASN15169 (GOOGLE, US),
Reverse DNS
Software
Golfe2 /
Resource Hash
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

pragma
no-cache
date
Fri, 15 Dec 2023 14:48:19 GMT
server
Golfe2
content-type
text/plain
access-control-allow-origin
https://www.searchenginejournal.com
cache-control
no-cache, no-store, must-revalidate
access-control-allow-credentials
true
cross-origin-resource-policy
cross-origin
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length
0
expires
Fri, 01 Jan 1990 00:00:00 GMT
/
tr.lfeeder.com/
43 B
293 B
Image
General
Full URL
https://tr.lfeeder.com/?sid=ywVkO4XKPNd7Z6Bj&data=eyJnYVRyYWNraW5nSWRzIjpbXSwiZ2FNZWFzdXJlbWVudElkcyI6WyJHLVRKUFpNS1NIRVMiLCJHLVMwOEpYNTEwRVMiLCJHLTlRUVBMMjEyWjQiXSwiZ2FDbGllbnRJZHMiOlsiMTMyNjk3MjkyNC4xNzAyNjUxNjk5Il0sImNvbnRleHQiOnsibGlicmFyeSI6eyJuYW1lIjoibGZ0cmFja2VyIiwidmVyc2lvbiI6IjIuNjEuMyJ9LCJwYWdlVXJsIjoiaHR0cHM6Ly93d3cuc2VhcmNoZW5naW5lam91cm5hbC5jb20vdGVjaG5pY2FsLXNlby9tZXRhLXJvYm90cy10YWdzLXJvYm90cy10eHQvIiwicGFnZVRpdGxlIjoiQmVzdCBQcmFjdGljZXMgZm9yIFNldHRpbmcgVXAgTWV0YSBSb2JvdHMgVGFncyAmIFJvYm90cy50eHQiLCJyZWZlcnJlciI6IiJ9LCJldmVudCI6InRyYWNraW5nLWV2ZW50IiwiY2xpZW50RXZlbnRJZCI6ImE4NjgyNmM5ZDBlZTFkMTIiLCJzY3JpcHRJZCI6Inl3VmtPNFhLUE5kN1o2QmoiLCJjb29raWVzRW5hYmxlZCI6dHJ1ZSwiY29uc2VudExldmVsIjoibm9uZSIsImFub255bWl6ZUlwIjpmYWxzZSwibGZDbGllbnRJZCI6IkxGMS4xLjYzM2U4NWI4NzVjNzI3MDkuMTcwMjY1MTY5OTMwNSIsImZvcmVpZ25Db29raWVzIjpbXSwicHJvcGVydGllcyI6e30sImF1dG9UcmFja2luZ0VuYWJsZWQiOnRydWUsImF1dG9UcmFja2luZ01vZGUiOiJvbl9zY3JpcHRfbG9hZCJ9
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
18.64.103.125 , United States, ASN16509 (AMAZON-02, US),
Reverse DNS
server-18-64-103-125.txl50.r.cloudfront.net
Software
CloudFront /
Resource Hash
b1442e85b03bdcaf66dc58c7abb98745dd2687d86350be9a298a1d9382ac849b

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:19 GMT
via
1.1 25cea41ad7d46d4db1c5c43579cc21ee.cloudfront.net (CloudFront)
server
CloudFront
x-amz-cf-pop
TXL50-P3
x-cache
LambdaGeneratedResponse from cloudfront
content-type
image/gif
content-length
43
x-amz-cf-id
dwUCm9qtwTJ8JPvZ7D5FW8A6Mgs_h7bcVCulI5S4BFFKjoQD3Dcenw==
edge
cmp.uniconsent.com/v2/
0
463 B
XHR
General
Full URL
https://cmp.uniconsent.com/v2/edge
Requested by
Host: cmp.uniconsent.com
URL: https://cmp.uniconsent.com/v2/main-v3.min.js
Protocol
H3
Security
QUIC, , AES_128_GCM
Server
2606:4700:3031::ac43:d3be , United States, ASN13335 (CLOUDFLARENET, US),
Reverse DNS
Software
cloudflare /
Resource Hash
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Request headers

Referer
https://www.searchenginejournal.com/
accept-language
en-GB,en;q=0.9
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36
Content-Type
text/plain;charset=UTF-8

Response headers

date
Fri, 15 Dec 2023 14:48:19 GMT
cf-cache-status
DYNAMIC
nel
{"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server
cloudflare
access-control-allow-methods
GET, POST, OPTIONS
content-type
application/json
access-control-allow-origin
*
report-to
{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=u3j%2BjTvFd2YltcUpL7ezi%2B90Y%2FgZL0SnJL%2F1b08zyxAmUZ%2Bz%2B4k9AlTrGSepABaSgO3fRHkoUpDN5YM8Nd%2F8nFDFC4CVSzfktUVu6JG6Ur%2FSup6h3pTLkHuc8TpiFXP0nAIMlwW58GowHz06LazcqAk%3D"}],"group":"cf-nel","max_age":604800}
cache-control
no-cache, no-store, must-revalidate
access-control-allow-credentials
true
cf-ray
835f7ca18b0f63f9-LHR
access-control-allow-headers
Content-Type
alt-svc
h3=":443"; ma=86400
js
www.googletagmanager.com/gtag/
261 KB
88 KB
Script
General
Full URL
https://www.googletagmanager.com/gtag/js?id=G-QMGPZ7WN5P&l=dataLayer&cx=c
Requested by
Host: www.googletagmanager.com
URL: https://www.googletagmanager.com/gtm.js?id=GTM-M2HM2DB
Protocol
H3
Security
QUIC, , AES_128_GCM
Server
2a00:1450:4001:80f::2008 Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),
Reverse DNS
Software
Google Tag Manager /
Resource Hash
25a003bc78e81e181deb2d9174522f3354a59bf6e635a467a8b80d475548645d
Security Headers
Name Value
Strict-Transport-Security max-age=31536000; includeSubDomains
X-Xss-Protection 0

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

date
Fri, 15 Dec 2023 14:48:20 GMT
content-encoding
br
strict-transport-security
max-age=31536000; includeSubDomains
server
Google Tag Manager
vary
Accept-Encoding
content-type
application/javascript; charset=UTF-8
access-control-allow-origin
*
cache-control
private, max-age=900
access-control-allow-credentials
true
cross-origin-resource-policy
cross-origin
access-control-allow-headers
Cache-Control
content-length
90312
x-xss-protection
0
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
expires
Fri, 15 Dec 2023 14:48:20 GMT
collect
region1.analytics.google.com/g/
0
45 B
Ping
General
Full URL
https://region1.analytics.google.com/g/collect?v=2&tid=G-QMGPZ7WN5P&gtm=45je3bt0v883690961z8836266955&_p=1702651698685&_gaz=1&gcd=11l1l1l1l1&dma=0&tcfd=10000&cid=1326972924.1702651699&ul=en-us&sr=1600x1200&ir=1&uaa=&uab=&uafvl=&uamb=0&uam=&uap=&uapv=&uaw=0&_eu=EA&_s=1&sid=1702651700&sct=1&seg=0&dl=https%3A%2F%2Fwww.searchenginejournal.com%2Ftechnical-seo%2Fmeta-robots-tags-robots-txt%2F&dt=Best%20Practices%20for%20Setting%20Up%20Meta%20Robots%20Tags%20%26%20Robots.txt&en=page_view&_fv=1&_ss=1&tfd=4162
Requested by
Host: www.googletagmanager.com
URL: https://www.googletagmanager.com/gtag/js?id=G-QMGPZ7WN5P&l=dataLayer&cx=c
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2001:4860:4802:34::36 , United States, ASN15169 (GOOGLE, US),
Reverse DNS
Software
Golfe2 /
Resource Hash
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

pragma
no-cache
date
Fri, 15 Dec 2023 14:48:20 GMT
server
Golfe2
content-type
text/plain
access-control-allow-origin
https://www.searchenginejournal.com
cache-control
no-cache, no-store, must-revalidate
access-control-allow-credentials
true
cross-origin-resource-policy
cross-origin
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length
0
expires
Fri, 01 Jan 1990 00:00:00 GMT
collect
stats.g.doubleclick.net/g/
0
45 B
Ping
General
Full URL
https://stats.g.doubleclick.net/g/collect?v=2&tid=G-QMGPZ7WN5P&cid=1326972924.1702651699&gtm=45je3bt0v883690961z8836266955&aip=1&dma=0&gcd=11l1l1l1l1
Requested by
Host: www.googletagmanager.com
URL: https://www.googletagmanager.com/gtag/js?id=G-QMGPZ7WN5P&l=dataLayer&cx=c
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2a00:1450:400c:c00::9a Brussels, Belgium, ASN15169 (GOOGLE, US),
Reverse DNS
Software
Golfe2 /
Resource Hash
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

pragma
no-cache
date
Fri, 15 Dec 2023 14:48:20 GMT
server
Golfe2
content-type
text/plain
access-control-allow-origin
https://www.searchenginejournal.com
cache-control
no-cache, no-store, must-revalidate
access-control-allow-credentials
true
cross-origin-resource-policy
cross-origin
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length
0
expires
Fri, 01 Jan 1990 00:00:00 GMT
ga-audiences
www.google.co.uk/ads/
42 B
107 B
Image
General
Full URL
https://www.google.co.uk/ads/ga-audiences?v=1&t=sr&slf_rd=1&_r=4&tid=G-QMGPZ7WN5P&cid=1326972924.1702651699&gtm=45je3bt0v883690961z8836266955&aip=1&dma=0&gcd=11l1l1l1l1&z=1948115352
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
2a00:1450:4001:82a::2003 Frankfurt am Main, Germany, ASN15169 (GOOGLE, US),
Reverse DNS
Software
cafe /
Resource Hash
ef1955ae757c8b966c83248350331bd3a30f658ced11f387f8ebf05ab3368629
Security Headers
Name Value
X-Content-Type-Options nosniff
X-Xss-Protection 0

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

pragma
no-cache
date
Fri, 15 Dec 2023 14:48:20 GMT
x-content-type-options
nosniff
server
cafe
content-type
image/gif
p3p
policyref="https://www.googleadservices.com/pagead/p3p.xml", CP="NOI DEV PSA PSD IVA IVD OTP OUR OTR IND OTC"
cache-control
no-cache, no-store, must-revalidate
cross-origin-resource-policy
cross-origin
timing-allow-origin
*
alt-svc
h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length
42
x-xss-protection
0
expires
Fri, 01 Jan 1990 00:00:00 GMT
id
id.crwdcntrl.net/
61 B
343 B
Fetch
General
Full URL
https://id.crwdcntrl.net/id?gdpr_applies=true&gdpr_consent=CP20-YAP20-YABEADBENAfEgAAAAAEPAACiQAAASZav-P_-2v7f8vXv0_bo8xq_77H33YhiDC-TZ9wLkXet-TgvvsJkaJt1bUwVMiQJduyCQgLaOROoipFC1RArXmPYxIM5AmaUT8DJAOYi9s6wELzfMxbH8gUzz_T-7u_zsn_7-7f9u7nnn6rPfu96_c7K4RE61z7aP_P-2iU392Q97_39_xfp6b_XJt-8mrf_69bf3rwAA&c=17533
Requested by
Host: www.searchenginejournal.com
URL: https://www.searchenginejournal.com/wp-content/plugins/abm-sej/includes/infinte-scroll/js/prebid/prebid-8.27L.js
Protocol
H2
Security
TLS 1.2, ECDHE_RSA, AES_128_GCM
Server
54.216.91.216 Dublin, Ireland, ASN16509 (AMAZON-02, US),
Reverse DNS
ec2-54-216-91-216.eu-west-1.compute.amazonaws.com
Software
Jetty(9.4.38.v20210224) /
Resource Hash
5076681c914e2a482c8bc4d4a743f7e7f85c329d0b745a8503af7e1fbaee4865

Request headers

Referer
https://www.searchenginejournal.com/
accept-language
en-GB,en;q=0.9
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36
content-type
text/plain

Response headers

pragma
no-cache
date
Fri, 15 Dec 2023 14:48:20 GMT
server
Jetty(9.4.38.v20210224)
content-type
application/json;charset=utf-8
p3p
CP=NOI DSP COR NID PSAa PSDa OUR UNI COM NAV
access-control-allow-origin
https://www.searchenginejournal.com
cache-control
no-cache
x-server
10.45.15.95
access-control-allow-credentials
true
content-length
61
expires
0
fpc
www.searchenginejournal.com/cvx/client/sync/
43 B
285 B
Image
General
Full URL
https://www.searchenginejournal.com/cvx/client/sync/fpc?id=pubcid%3A898dc952-00f5-4f91-a371-816030545045
Protocol
H2
Security
TLS 1.3, , AES_128_GCM
Server
136.243.171.225 Ludwigshafen am Rhein, Germany, ASN24940 (HETZNER-AS, DE),
Reverse DNS
edge.presslabs.net
Software
nginx /
Resource Hash
b1442e85b03bdcaf66dc58c7abb98745dd2687d86350be9a298a1d9382ac849b

Request headers

accept-language
en-GB,en;q=0.9
Referer
https://www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt/
User-Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.109 Safari/537.36

Response headers

pragma
no-cache
date
Fri, 15 Dec 2023 14:48:20 GMT
server
nginx
accept-ch
Sec-CH-UA-Platform-Version,Sec-CH-UA-Arch,Sec-CH-UA-Model,Sec-CH-UA-Bitness
x-presslabs-stats
desktop
vary
Accept-Encoding
content-type
image/gif
cache-control
no-cache
content-length
43
x-request-id
5c99ff1b855306376490f20cf665a00d
expires
0

Verdicts & Comments Add Verdict or Comment

527 JavaScript Global Variables

These are the non-standard "global" variables defined on the window object. These can be helpful in identifying possible client-side frameworks and code.

object| 0 object| 1 object| 2 object| documentPictureInPicture string| aj_p1 string| aj_p2 string| ajaxurl number| tkrand number| tkrand_abcd boolean| mobile_bool boolean| cmpready boolean| smobile boolean| spod_category boolean| ssponsor_cat boolean| ssingle_page boolean| sinfscroll boolean| stakeover boolean| shome string| sguides boolean| s_iscategory boolean| s_archive string| s_author boolean| g_algo number| s_author_id boolean| s_news number| pp_id number| is_long_article string| s_word_count string| is_re_re string| custom_targeting string| scategory string| s_primary_c string| has_summary function| sappend function| sappendHtml function| sparents function| sadd_event function| saddClass function| stoggleClass function| sremoveClass function| sremove_element function| shasClass function| sisWindow function| sgetWindow function| soffset function| setAttr function| getAttr function| removeAttr function| s_selecor_all function| s_selecor function| sej_isRetina function| sej_getUrlParameterByName function| sej_utm_var function| striggerEvent function| sajax_call function| sajax_call_post object| opts function| __gaTrackers function| sloadScript function| script_loader function| slog function| abm_sej_setCookie function| abm_sej_getCookie function| abm_delete_Cookie function| sdeleteAllCookies function| sok_callback function| serror_call function| sdelay boolean| supportsPassive boolean| b_webp boolean| spromise boolean| ss_u string| sopp string| sglob_country string| sglob_cont boolean| s_tab_active string| s_is_bot boolean| s_headless boolean| supportsImport string| __unic_cmp_id boolean| __unic_cmp_prod string| __unic_cmp_host function| __unic_loadapp string| sej_dev_site boolean| is_adblocker number| swindow_width boolean| s_tablet number| randomInt number| preb_t number| preb_sync number| lazy_load_ad object| apstag object| sej_posts_shares boolean| newtrck boolean| sga_t_u string| sga_no_track_reason function| sga4 object| sdimensions object| presslabs object| s_glob_p object| sej_show_ads_exp object| sej_no_show_programmatic number| cis_current_post_id number| cis_source_post_id string| cis_ajaxurl boolean| cis_mob string| SEJ_TAKEOVER_NAME string| SEJ_TAKEOVER_ADV string| SEJ_TAKEOVER_TOPIC object| doc string| s_bottom_wsource string| s_bottom_form_ebook function| sej_tcf_deny_ga function| __tcfapi function| __uspapi function| __gpp_addFrame function| __gpp_stub function| __gpp_msghandler function| __gpp object| tk_allow_countries object| disallowed_referrers string| referrer boolean| isReferrerDisallowed object| disallowed_agents string| uagent boolean| isAgentDisallowed function| sej_is_ref_units number| PREBID_TIMEOUT number| PREBID_TIMEOUT_REF object| pdKeys object| pbjs object| googletag object| dynamicSlotsDone function| s_isIE function| sej_get_ads_utm function| sok_i_callback function| serror_i_call function| ss_push_inf_slots function| sej_sstr function| aps_refreshBids function| initAdserver function| initDynamicSlotPrebid function| sej_trefresh string| u_s_i boolean| pd_s string| pd_s_i boolean| shave_ads object| usp_ccpa string| usp_ccpa_str boolean| usp_ccpa_bool boolean| is_blockthrough number| p_refresh_floor number| p_norefresh_floor number| s_max_ad_calls_minute number| if_sej_domain number| if_loc_dom object| sej_ref_ad_units object| ad_slot_alias2 object| ad_slots object| ad_slots_desktop object| adUnits number| P_T_REFRESH number| FAILSAFE_TIMEOUT boolean| ver_get_param string| trck string| utm_track string| id5pdString string| s_sovrn_lb_top string| s_sovrn_lb_foot string| s_under_f string| s_under_r string| s_under_f_i string| s_mid_p_i string| s_under_r_i string| s_loop string| s_under_p_i string| s_under_post string| s_hello string| sh_t string| sh_t_und string| sh_t_loop string| g_loop string| g_under_f string| g_under_r string| g_guide object| s_s4_sizes object| s_leadb_sizes object| s_leadb_size_sticky object| s_leadb_cont object| s_sid_sizes object| s_sid_sizes_ex160 object| s_sid_sizes_2 object| ix_large object| ix_large_top object| ix_sidebar object| ix_large_2 object| ix_large_cont object| ix_under_post object| ix_hello object| s_m_stikcy object| smart_incont object| smart_loop string| pm_u_fimg string| pm_u_incon string| pm_u_rel string| pm_u_loop object| std_bp object| std_bp2 object| adUnits_desktop object| adUnits_lazy object| adUnits_final object| adUnits_desktop_final string| sinclude_home string| sinclude_single string| sinclude_category string| sgalgo_inc string| sexclude_takeover string| sexclude_global string| sinclude_all_guides number| kk string| sstr string| s_str number| vv number| ii object| customConfigObject object| apstagSlots object| scontent_size object| suidsobj boolean| fallback_adsense object| unicj function| __unic_start object| UnicI object| pbjsChunk object| _pbjsGlobals object| mnet object| sonfig function| appendAliases function| cis_get_size_by_slot_id function| cis_get_lazy_ad_size function| s_sleep function| lazy_ads_make_request_wrap function| s_ad_trigger function| lazy_ads_make_request function| sej_search_in_array function| slazy_ads function| observer_handler function| isPastViewport function| sej_ad_destroyer function| s_is_ads_libs_ready function| sready boolean| delay_ads_load_anchor_url number| timeout_cmp function| inf_ads_load_main_init function| s_scroll_init function| inf_insert_sidebar_div function| inf_init_sideabr function| inf_init_sidebar_160_600 number| w_height number| w_width object| goog_adslots boolean| first_scrl number| scroll_const boolean| goog_year_change number| total_in_cont_ads number| max_sidebar_ads number| s_number_of_ads_loading number| balance_load string| ads_per_distance_class number| ads_per_distance object| sizes_db_desktop object| sizes_db_tablet object| sizes_db_mobile boolean| observer_api boolean| s_ga_err_detail_unique object| soptions number| destroy_ad_margin number| destroy_interval object| destroyObserver boolean| ifs_scrolled object| ws_form_settings object| wsf_form_json_config object| wsf_form_json object| wsf_form_json_populate object| post_obj function| sej_scroll_dir string| scroll_dir number| ss_lastScrollTop2 function| $ function| jQuery function| s_trigger_pixel function| close_hello_bar function| close_display_bar function| display_ad function| click_hello_bar function| h_init_main_bar number| h_show_time boolean| h_worked boolean| main_bar_first function| sej_ttk_impressions function| sej_tk_start_tracking function| Carousel function| Fancybox function| Panzoom function| sej_simple_lightbox function| sej_gallery_lightbox function| touchstartListener function| mouseoverListener function| mousedownListener function| mouseoutListener function| mousedownShortcutListener function| isPreloadable function| preload function| sej_cdet function| sej_cdet_ws function| _extends function| _typeof boolean| is_retina undefined| len number| pixel_limit object| lazyLoadInstance function| LazyLoad object| sitems function| sej_save_regsource function| sej_get_user_source function| sej_get_user_first_source function| sej_user_interests function| shsow_shares function| sej_is_disposable function| share_buttons_pop_click function| sej_sharepop function| getMobileOS function| menu_tab_open function| show_search_icon function| hide_search_icon function| sej_recaptcha_solve function| scaptchacallback function| SEJ_contains_letter function| SEJ_CheckIsValidDomain function| SEJ_CheckIsValidPhone function| sej_check_website function| SEJ_ValidURL function| sej_is_selected function| sej_is_checked function| SEJ2017isValidEmailAddress function| SEJ2017is_Valid_Syntax_Email_Address function| sej2017_shorten_long_digits_K function| sej2017_getFirstKey function| sopt_out function| sopt_in number| checkb boolean| singopt object| elem object| el boolean| recaptchachecked object| wsf_form_instances function| wsf_form_init function| sej_insert_txt_ad function| s_pop_init object| wpadmin number| k object| name_inputs number| sej_thumb_h object| post_bottom number| post_bottom_pos number| sw_h object| sinf_badge number| ss_limit_up number| ss_scroll_dictance number| ss_limit_down number| sticky_nav_tresh object| post_toc object| __bt object| __bt_intrnl object| __bt_tag_d object| __bt_tag_am string| the_permalink string| subs_email_tmp string| s_ga_category_wsf string| s_ga_action_wsf function| __unicapi object| _aps boolean| apstagLOADED object| apscustom object| ggeac object| google_tag_data object| google_js_reporting_queue number| __adb boolean| __bt_already_invoked object| lotame_sync_16576 function| ha undefined| google_measure_js_timing object| __unic_tags_loaded object| hadron boolean| __halo_loaded__ object| regeneratorRuntime object| ID5 object| __id5_instances function| clearImmediate function| setImmediate object| ats function| lotameIsCompatible function| sync16576_aa function| sync16576_c undefined| sync16576_d undefined| sync16576_ba undefined| sync16576_e function| sync16576_f object| sync16576_h function| sync16576_ca function| sync16576_j function| sync16576_da object| sync16576_ object| sync16576_ga object| sync16576_v object| sync16576_oa object| sync16576_xa object| sync16576_ya function| sync16576_a function| sync16576_b function| sync16576_g function| sync16576_i function| sync16576_k function| sync16576_l function| sync16576_m function| sync16576_n function| sync16576_o function| sync16576_p function| sync16576_q function| sync16576_r function| sync16576_fa function| sync16576_ea function| sync16576_s function| sync16576_t function| sync16576_u function| sync16576_w function| sync16576_ha function| sync16576_ia function| sync16576_y function| sync16576_ja function| sync16576_z function| sync16576_A function| sync16576_x function| sync16576_B function| sync16576_ka function| sync16576_C function| sync16576_D function| sync16576_E function| sync16576_F function| sync16576_G function| sync16576_H function| sync16576_I function| sync16576_J function| sync16576_K function| sync16576_L function| sync16576_la function| sync16576_ma function| sync16576_na function| sync16576_M function| sync16576_N function| sync16576_pa function| sync16576_O function| sync16576_qa function| sync16576_ra function| sync16576_sa function| sync16576_P function| sync16576_ta function| sync16576_ua function| sync16576_va function| sync16576_wa function| sync16576_Q function| sync16576_R function| sync16576_za function| sync16576_S function| sync16576_T function| sync16576_U function| sync16576_V function| sync16576_Aa function| sync16576_W function| sync16576_X function| sync16576_Y function| sync16576_Z function| sync16576__ function| sync16576_0 function| sync16576_Ea function| sync16576_Ba function| sync16576_1 function| sync16576_Da function| sync16576_Ca function| sync16576_2 function| sync16576_3 function| sync16576_4 function| sync16576_5 function| sync16576_Ga function| sync16576_Ha function| sync16576_Ja function| sync16576_Fa function| sync16576_7 function| sync16576_Ia function| sync16576_La function| sync16576_Ka function| sync16576_8 function| sync16576_6 function| sync16576_9 function| sync16576_Ma function| sync16576_Na function| sync16576_Oa function| sync16576_Pa function| sync16576_$ function| sync16576_Qa function| sync16576_Ra function| sync16576_Sa function| sync16576_Ta object| PublisherCommonId object| tk_obs_options object| tk_slots number| tk_l object| observer_tk object| entry string| p_title object| au object| dataLayer object| google_tag_manager function| ldfdr object| fs function| onYouTubeIframeAPIReady object| gaGlobal object| __gaConnectorEventsEmitted

13 Cookies

Domain/Path Name / Value
www.searchenginejournal.com/technical-seo/meta-robots-tags-robots-txt Name: unic_usp
Value: 1
.searchenginejournal.com/ Name: lotame_domain_check
Value: searchenginejournal.com
www.searchenginejournal.com/ Name: _lr_geo_location
Value: GB
.searchenginejournal.com/ Name: _gcl_au
Value: 1.1.136736552.1702651699
.searchenginejournal.com/ Name: _ga_TJPZMKSHES
Value: GS1.1.1702651699.1.0.1702651699.60.0.0
.searchenginejournal.com/ Name: _ga
Value: GA1.1.1326972924.1702651699
.searchenginejournal.com/ Name: _ga_S08JX510ES
Value: GS1.1.1702651699.1.0.1702651699.0.0.0
.searchenginejournal.com/ Name: _ga_9QQPL212Z4
Value: GS1.1.1702651699.1.0.1702651699.0.0.0
.searchenginejournal.com/ Name: _lfa
Value: LF1.1.633e85b875c72709.1702651699305
.searchenginejournal.com/ Name: _ga_QMGPZ7WN5P
Value: GS1.1.1702651700.1.0.1702651700.60.0.0
.searchenginejournal.com/ Name: _pubcid
Value: 898dc952-00f5-4f91-a371-816030545045
.searchenginejournal.com/ Name: _pubcid_cst
Value: 1yxjLEgssw%3D%3D
.searchenginejournal.com/ Name: panoramaId_expiry
Value: 1702738100814

Security Headers

This page lists any security headers set by the main page. If you want to understand what these mean and how to use them, head on over to this page

Header Value
Content-Security-Policy upgrade-insecure-requests
Strict-Transport-Security max-age=31536000; preload
X-Content-Type-Options nosniff
X-Frame-Options SAMEORIGIN
X-Xss-Protection 1; mode=block

Indicators

This is a term in the security industry to describe indicators such as IPs, Domains, Hashes, etc. This does not imply that any of these indicate malicious activity.

a.ad.gt
ad-delivery.net
ad.doubleclick.net
api.btloader.com
api.ipify.org
ats.rlcdn.com
bcp.crwdcntrl.net
btloader.com
c.amazon-adsystem.com
cdn.hadronid.net
cdn.id5-sync.com
cmp.uniconsent.com
config.aps.amazon-adsystem.com
geo.privacymanager.io
id.crwdcntrl.net
id.hadron.ad.gt
openfpcdn.io
region1.analytics.google.com
region1.google-analytics.com
sc.lfeeder.com
secure.cdn.fastclick.net
securepubads.g.doubleclick.net
stats.g.doubleclick.net
tags.crwdcntrl.net
tr.lfeeder.com
www.google.co.uk
www.googletagmanager.com
www.searchenginejournal.com
104.237.62.212
13.227.219.68
13.32.99.97
130.211.23.194
136.243.171.225
142.250.181.230
18.64.103.125
18.64.119.20
18.64.79.46
184.30.211.26
2001:4860:4802:32::36
2001:4860:4802:34::36
2600:9000:2261:3200:4:d7e1:700:93a1
2606:4700:10::6816:3556
2606:4700:10::6816:445
2606:4700:10::6816:4bd8
2606:4700:10::ac43:17ea
2606:4700:10::ac43:246e
2606:4700:20::681a:246
2606:4700:3031::ac43:d3be
2a00:1450:4001:80f::2008
2a00:1450:4001:829::2002
2a00:1450:4001:82a::2003
2a00:1450:400c:c00::9a
54.216.91.216
99.84.149.113
99.86.4.128
013eec022b83c1f01c2127108b2724d94f43a2c91ac337a3d317632739117ff0
013fc39efb38a28d8eccab58189059646847bc5c54e1c4b637e874b6109ee0ef
022c491ef985ac446182294ceb47e595ecdbb537300c51e2ec10fc47387c62f7
0247df6b9220981fc47473c27043ad09602f542a58d563f44ae5a4e91b827329
053cb3f560d39447ce5eae1f319e68e76c56c67bcd1727db08c6cd3a480be51f
06b99248a163333e36980a6cfb756f1a7de60fa49517162b87b1a44d5d48f844
0755f702a81f095a3b3814801f57841ba195a5a0e955b0168ff0a4007127e925
0bb059e797c405edc79fe7bc59a52b17a5a3fd3511adaba3a56a1cf543c973aa
0c9fb498a29dd3d10f8a6ade5bfb1b38aa0f91be670eb0c538413963d1bc06ac
0f8b5718fb74df9076bc80ad9c0f85f387438f22989f6e33cbb5853ec96c6cd7
10dc140898ff0dbc97793f8f1f805d8f93b81cc0d80aa65d6b3040d4c71698fc
111d4518d77b165368c37669f563a1d435e3766dda9544ae28c7ed965e05afb6
12ba93db33de679d443dc28aee4a2190b580b8ad3fc53216d5bb2678d4e17f29
1632154cd5abe8555eb15315d6b3d0b1f40c26d08e6ce78f0160224de320ac4f
16abe109580561cc0ef661ba3bc4c5c212998d4f4c9cc0e693e3eba57011e907
1756fb46aa1a7fe26e8c57328db54497ef70236895be1a7160718bb772abe23b
18c4f9d98f61be71d13dbb438be923a05012c58390c394919a98c6ef7308d93c
1b9d62fca3818f9b4d6fbe625e0b7adeb32c98a9eac289519f6bf7c165571c12
1d457324cbe10e73b26ccfbbe9b797e6972b85d4ba1419c4ab233b0ec427fcc1
1f006bec36d62d033e3a64bd87ec8de0cdff7541f0e6950454d4ef866c042ee6
1f0c0d5a173bd397b39ee432ffaf04528f62188debc93b819ed6c36521e59b5d
1fcc2492b4655a83c76b09d6ad4cab1fc73668994e3f2d01dcfcd384fcfa49ae
20f405357a6a92ff60139fd91e0dc0c0133d9da8efa70ae530e50e04786667d0
2289485ae05bbe8f0ec23f821aff7c01e1ebf720bb949d221790b68dba188408
2365cc11ef3d43f265b848c7164e5487c7a49d6af06c2938ac9272c8d91fc1a2
24ae7b9ac5220f301dcfc8032ba2e78994ef99a7d859c7c343c5829259e318af
253587dbc60fd8e8074a091f8e40f8259fe35d2a1a2e6671b845f1e4718fd548
25a003bc78e81e181deb2d9174522f3354a59bf6e635a467a8b80d475548645d
2868c587f51b3ff82a480f29218ce4b911a1cea8e4f2815ff5112ff56bb4362e
28fb91355106692115c5ca2c04cd15b441692c106ed07bd4cd398d14568b8f0f
292ab40cd9c46838d7c5450d5b159fbc62bf4d5663ae95512d599aaa9925ae89
2bc0406454463b7cca251018ae904ae21c02f0e835acd296c5a8229074756245
30850680c0f24c91b3130c1563e324fd245ac50f21bd4d5cc7010429f606b744
31a87e09bc3fc7ccb2149b418299ecae339b11000c5c0c4f0ff574cbe8b2318d
340a49ba6b22c47b01b4aa9e42bc9b6b75f0a3b7938724ccf9347579cd0a0668
34584471249c18eec831bf83fa69311328233f05a87e909ccba4d9623620a1db
34cf4cbc97aba634d40287ae9d58896d84105879cd2173ad0361662222f94aee
360ec0a5e7fd9eb35367277e9257aa6378fedb7bbf0bc53ad8e3886820086b09
38c872afb161a43f9bcfad5edefd2d2ee6583001be855fbee5be6069564523bc
3aa68b6ea45ed5e1eaaf779a2377f7689dc531620ee9d57a6c0f6444ad11f7e3
3da9f17f868d54688d41272be758db97faba155afdf559ade101edd299fb2b62
3edc0d7881798bbf380ea9dd671c8f90b3e9fd091a73d9fe22c59deb5994ddbe
404946964b5a4084de4bdbeb860c1679ee825c1436e6269fcf48f889952394f7
4117699ce11d0dab6c19cf61dd4ca27aaa826d2807c3462a624daa20dae58a95
4304ad9301017a03d77d588218c73d2b5fbc0a292138994c023a10aed3196740
43f804d38a294c6df1ce8ee64fb95ad0ff5a8d6d5685d9537df02212668a1dff
450f3ba4e47ee174bd9692b396f264b907d37d2528f53911760f3d0edb785f7e
46e1a5317649fd50f6f2cd67bbb8efb9a176e613d75442987855ba686b7682bf
486dbb49fb36a8b8e7bb750b738fab6a2bd3aee598b3a71d8da405245447dd0f
494e6ad3ea844634436d169312c020a27761bd7fc0ae20649624296655c41cfb
4b3e23e828653d43cefd0d3b5c4e477c5aa9ab6350e66cdf54d1d658b1ad6db8
4d22abbb9a6a14f22b41d50b8e97dcad4a9289af3df46a615b0c70f586d645a3
4d3c300c1cd89393c7f945c06656981e3ac1c034f59996affcd1062a3092f40c
4dc868e5e30c80252a63341c4a0b7b692f79503c78149d5deca3448210752421
4fca7e9d2305082e22ac89090f8b7e9488602b6b4ef27dd30c9342f9c36b0c1e
4feba251a69abc0926f05804e759752ad346892f397ffdac23ab18a7b5b009e8
5076681c914e2a482c8bc4d4a743f7e7f85c329d0b745a8503af7e1fbaee4865
53b4ab18872c599c620b3d329d3894f9c2973d3339b4ca4679e6afd4215a01f0
5485428ad75e744e1ca665f207db700ed41a3c0a9244bb52637252a14ae841eb
57b19a7a79fcf081749d34fdd3a96cbc11419416b760a3be213aaf509097de17
59c66051ef0f633b25e7cdf631bbd1c9289a000e55f18bac9373f231c9bdaa5f
5d55ce9c3ac7a5f37a38813a2deed310abde9e63d0d36912e2e5e59751ca66c7
5da4ec024f5d8814b0f272f9e9ba8457e0c1615c7e25a4fda2dcf550eb46efe3
6095256b5d03a355c7dd87945ee5022f7a17135e2cba4f28834598cec0b09238
61fe6af974028ea3358d36fa08e7b5a7df61010e4e91f2ff33914012a24084fe
62901637b079ad004302620acc29667e05931a0bf2f8f8f0a992f863b7a0febd
63fa00bcc8093c9fc6652d02aebfe2e54e99ebad34954f877bdb51e655bdc5e2
6c1056b1b1182ed0a3532a6fdc40375191c21146b8387468c8561e317483395f
6c76972839a5126e9e07e4958eb6b5ed3ec40985e62564c9d31adcac47486da6
6e122ad3e684634a13a0c1cca44c145bfedd3ed864d06fae1d2f2253a1d035e2
72790a1735404379453c3f2c9f095b3c326f10c6efb14485931bac45075afbb9
7354cd99c980e411b94fb6ac1531864b79360a6eae38da19300316344c1af71b
76895c9b99fdb460c5ad04c00aa06c2d4e47aab67a8b98520f2adf20dacde846
7713183ba1a38b1ea2be2d5f7d3d49dab7b8d468cf78a603e6517ffbd1f33d59
79015be0ad1b860341c8a74cc4311ba5ea9b1502f467dc60b26894bf98b69f45
7acca53b6e2ee3f5fcbe256dde41cd1ef8906df5c998c0130830bbb37fe3fbca
85b0fe439768c208c0b3d9157178b8a1ac48fba8a191a3d8d7f1c3c1dcb1dcbd
865519f76b49f44e52ea431a295b39c46cdb31ce47822104fad3c4e2e80cb4e7
8a78cef0b7848d146a9983a7ec0e37c321eef8a01d5847caa7483e5ecb86be04
8af8ed19513c17a73f34a27b342003a52850988e6aa67c97c808f340d5f09878
8b9180ed5e3b5e2d1acd5e2215070d59a589eae5de6ccd840e9e700acb71dcec
8c9bf876364a2e03c7f4e18103df82d248ffbd5bada31d3ecb4fd86936f22fa5
901a0b166ebe1970fd7e9d6d1c85791718ab530f1c63d293e9192063cebd20bf
917106b3d05f0bda89390297ea5672b298a0bf21bd494a53ba930c235522dd94
977884b3a28fd132dd91138134469596a5af3420ace9799253947e557e4a7483
98aabe4d6eacbd5bb3bffeeb71a0946e9a7241984ef448a981d8f64618b60453
999abb9809fe567abcda124de9ad2962f0f66c67c029ed96c0c629bb4c1d4878
9a1eb559f60a9f2344882c89cc97365185663d3507382c35d77a30ad5bfda98a
9c1a2fb00222ceeef376367653f9de6f386eff7ba9793eec110fd7be28421aca
9c2c86e88fe2074f9ddbd073add93e47208ae95d2947deb81e4da32c983b01d5
9ea87ca1966825417ae6feefde236700033bd308670bcc448d7094ac3cdacb7a
9f46649ea544819982ea288c6f386dd67d46da0f453f95da542196372b79731e
9fac1c1beffa703d282e759e514fac5f368fbacfb6fedd785c968dd191098bfe
9fd122da02d8e041cd491f078a0fddb3986846255c5a0fc64cbf37eddaeab0e7
9ff3d76471ddf3e6dc85d985ac7911883108c186d71bca9fdaf4524de72ed421
a171e64ec4287990b354e03ec80aa0b83e9c01be406cf9af7c232ec40439e7e5
a2aa2577c105dab138246b4e0a1f575b3c92c30d5aced108d3f73897bd46823f
a3cf00c109d907e543bc4f6dbc85eb31068f94515251347e9e57509b52ee3d74
a6333929a02027e96ecf154218e5fc09e7e64e629c4284d98766878a09d93988
abbfa9c5b2e5ccdcd77eed22c4862ddf1b0d83fde488da934fe67153780ba212
abd2c31c501f8efcd7dcb8bfda286fd337d79183b340d7fd1bc81a37f8d8a1cc
b1442e85b03bdcaf66dc58c7abb98745dd2687d86350be9a298a1d9382ac849b
b81b00bc7e76a7bb3b4c63f58208f092f0dee448ef77b1a117822fa9dbcf0e89
bf0808bd9c80866718bc07d9445c6fdadadabe21cc341d4fe156a1b09708d3fd
c02addcf940dcfd73293ccb530238f1f1737d33a6f61f324ef09ad1c69fcf594
c4abf725ca2c888df89bee0a81a50fc7dfc69777cec49ef80ceef97123983f47
c626dabe59f1e78ba16b99eec8c8291f67ce15788b7556911f68b9be562ac5bb
c7de47a055d9437f1e8ba6b9a6f2c33def0c2f49614305cf849d274a93c59f65
c84978335d23fa8e42836a32463987ccff0e9ebe9ad4429235f6a6da9bf19a96
c8ae451dc07e70ed8d9e22c8a12359517d8eeab96b4b16b036f129a28508a980
cf4724b2f736ed1a0ae6bc28f1ead963d9cd2c1fd87b6ef32e7799fc1c5c8bda
cf79a232f8fe7bd21ac01c89f6e1c3774e8914bf8ae900ae979877504662699d
cfc67cc35c76c672a4b35e86146752a3d437c8e6cf82309d21ffd4c91cb4fd7c
d571826e6090eb9ece1e287d9b726b1f63d2e43a3a84a4d17ce466468d14be6f
d632f1fd8fd7e6f6a9fbbdf2ad865c79432d41f8e203db015dfb473005243d95
d63969cac377eba8dc163b0ba842d53d263a6bbb31f1a6e59f3314da8d02de11
d7dd0007fc5d64b48c0fd0b26382a8c93b93ca82a9b32a7244cc73f63ba5393c
d7f9e18543446ef3f97b128bb12bc097732e3889e98a4fcc6d98801753b4e46d
d888d58ad34eaf2c795253ddc129cd07e5d7904bf6583614a4338e21901cd4c0
d961b08e4321250926de6f79087594975fe20ad1518de8f91eb711af5d1a6ef8
daa5cbff105aa4dd98b3982e70a45b7668f394e78b3e5b8a8a2ed470fa83c9c0
db3a067eb9d66b2baef7f7019877cae166bfd30d4cf801a244d1cc8ade33e61e
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
e3d695104174ae7c3034322cd7cd822375f51a1f5b0c121f3949ef38b6371353
e5429ce4c488099db846eb6568ffd7965bca538d9a28cec723bf561c372880b3
e7c302b5ae8fea99e6432528806e19f8a6fce4ab1c386f58f4daf8d10dba6d7a
e98439db943f5748a9bcc5f687bb3b66b3c50f9b77cf3d4eb4ad2fd53aed1258
e992ee382f729e845f3808b58e9af88bd5f8a49b2ad49f7eaa4b6da0074c8cb3
e9dfb181b7284e1d5674c9497da7aa98db42987a389b12f57b344285889bf893
ea7f8924b70229fe13fe0587bf81b59fa5a235690c936950f09cd0de156e013e
eb76d7c54772efe035bcd270556fc754f63087b700ee4ee5c99ce242f41b9d30
ee7300345a6db37b152f33f268dbc82862cc88242fa6ccb7f0b4b35bc350fecd
eef945f7b56946036572f7b2f4067b0c3c139440720704f39f1e25e7e250a57f
ef1955ae757c8b966c83248350331bd3a30f658ced11f387f8ebf05ab3368629
f203715aaaa622db1449a0b2f5695456d5d116f2464c4b144145f041d4a8b949
f50db6abd4c6f1298f381d938cee3dbc3fc2efd813723b5cbe62e2b215f036cc
fb19521eeb71c0b7bcdcbf3f457b81aac65f795eb7bda13b4c305b1f23ffa740
fb9cbcc030c3111a1c91b31feab02da45e55a4440b784cb6b4a4f743cd99edf7