How to Create XML Sitemap and Robots TXT for Your Website SEO

Source: Source jejaknesia.com

www.jejaknesia.com - Sitemap.xml is a file that contains a list of URLs on your site, helping search engines like Google to index those pages more efficiently. Meanwhile, robots.txt is a file that tells search engines which pages are allowed or disallowed to be indexed.

Why Sitemap.xml and Robots.txt Are Important for SEO?

By having both of these files, you give clear instructions to search engines about your site's structure, ensuring important pages are indexed and preventing indexing of unwanted pages. This can increase your site's visibility and ranking in search results.

How to Create Sitemap.xml in Blogger

For Blogger, the sitemap is generated automatically, so you just need to submit it to Google Search Console. Below, we will outline the steps. Meanwhile, to create sitemap.xml on a PHP-based site with hosting, follow these steps:

1. Open XML-Sitemaps.com
2. Enter your blog URL (for example, https://yourblog.blogspot.com) and click "Start"
3. Once the process is complete, download the generated sitemap.xml file
4. Upload the sitemap.xml file to your blog's root folder using FTP or File Manager in cPanel

How to Create Robots.txt in Blogger

To create robots.txt in Blogger, follow these steps:

1. Open your Blogger dashboard
2. Select the blog you want to configure
3. Go to the "Settings" > "Search Preferences" menu
4. In the "Crawlers and indexing" section, click "Edit" under "Custom robots.txt"
5. Enable the "Yes" option
6. Enter the following code:

User-agent: *
Disallow: /search
Allow: /
Sitemap: https://yourblog.blogspot.com/sitemap.xml

Replace https://yourblog.blogspot.com/sitemap.xml with your actual sitemap URL.

If your site uses PHP, you only need to create a robots.txt file. You can use a text editor like Notepad or TextEdit. Here is an example of the robots.txt file content:

User-agent: *
Disallow: /admin/
Disallow: /login/
Allow: /public/
Sitemap: https://www.yourblog.com/sitemap.xml

After creating the robots.txt file, save it as robots.txt and upload it to your website's root directory.

Adding Sitemap to Google Search Console

To ensure Google indexes your site correctly, add your sitemap to Google Search Console:

1. Visit Google Search Console
2. Add and verify your site if you haven’t done so
3. Select your site from the property list
4. In the left menu, click "Sitemaps"
5. In the "Add a new sitemap" section, enter sitemap.xml and click "Submit"
6. Repeat the same steps for sitemap-pages.xml

References: 

• Creating and Submitting a robots.txt File https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt?hl=id
• Creating and Submitting a Sitemap https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap?hl=id