Below you will find pages that utilize the taxonomy term “S-Thom”
August 2, 2026
Create Robots.txt
Version updated for https://github.com/s-thom/create-robots-txt-action to version v3.0.0.
This action is used across all versions by 6 repositories. Action Type This is a Node action using Node version 24.
Go to the GitHub Marketplace to find the latest changes.
Action Summary The Create Robots.txt Action generates a robots.txt file from various sources, including existing files, cloudflare API tokens to fetch bot categories, known agents API tokens to fetch user agent categories, and allows users to specify allowed and blocked bot names. It automates the creation of a robots.txt file for web scraping and SEO purposes by appending allow rules for all unspecified user agents if requested.