# robots.txt for 糖心传媒_糖心vlog传媒_糖心Vlog官方 # 本站允许所有搜索引擎蜘蛛抓取全部公开页面 User-agent: * Allow: / Disallow: User-agent: Baiduspider Allow: / User-agent: Baiduspider-image Allow: / User-agent: Baiduspider-video Allow: / User-agent: Googlebot Allow: / User-agent: Googlebot-Image Allow: / User-agent: 360Spider Allow: / User-agent: HaosouSpider Allow: / User-agent: Sogou web spider Allow: / User-agent: Sogou inst spider Allow: / User-agent: Bingbot Allow: / User-agent: YisouSpider Allow: / User-agent: Bytespider Allow: / User-agent: TianWangSpider Allow: / User-agent: Applebot Allow: / # 抓取频率建议 Crawl-delay: 1 # 站点首页 # https://www.tangxinmedia.cn/