Funky URLs are being generated

#1

Hello,

I noticed the articles that sits in articles directory get generated many times with a different number of slugs stacked on top of each other. To better demonstrate what I mean, here is what exactly happened since I’ve added a new article with /music-for-youtube-videos/ URL slug - yes, they are in google just like this which very likely hurts my SEO.

02 PM
02 PM1177×383 48.3 KB

I have no idea why is this happening but perhaps some of you might know the issue.
Maybe that has something to do with the fact that besides article pages, I also have other pages with data pulled from google spreadsheets.

My stack:

ruby "2.2.3"

gem 'middleman', '~> 4.2'
gem "middleman-blog"
#2

One more thing - I noticed no matter what URL I type in after mydomain[.]com, I basically don’t get 404, instead, I am landing on some random page that has a slightly broken styling. So I assume something is pretty wrong with my setup, just don’t know what.

#3

What web-server are you using and do you use some redirects or similar functionalities? If any URL gets a hit, this means the problem is probably not inside Middleman setup, but http-server setup.

#4

Thanks for your reply! I am using NGINX, this is config for this website (it’s not a root/main website):

server {
    listen 80;
    listen [::]:80;
    server_name www.MYWEBSITE.com MYWEBSITE.com;
    return 301 https://MYWEBSITE.com$request_uri;

}

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name www.MYWEBSITE.com;
    return 301 https://MYWEBSITE.com$request_uri;

}

server {

    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    server_name MYWEBSITE.com;

    root /home/yuumee/www/MYWEBSITE.com;
    index index.html index.php;

    access_log  /var/log/nginx/MYWEBSITE.com.access.log;
    error_log   /var/log/nginx/MYWEBSITE.com.error.log;

    error_page 401 403 404 /404.html;
    charset utf-8;

    include snippets/ssl-MYWEBSITE.com.conf;
    include snippets/ssl-params.conf;

    location ~ /.well-known {
       allow all;
    }

    location / {
      # try_files $uri $uri/ /index.php$args;
      try_files $uri $uri/ /index.html /index.php;
    }

    #location ~* \.html$ {
    #    expires -1;
    #}

    location ~* \.(css|js|gif|jpe?g|png|ico)$ {
        expires 30d;
        add_header Pragma public;
        add_header Cache-Control "public, must-revalidate, proxy-revalidate";
    }

    location ~*  \.(pdf)$ {
        expires 30d;
    }

    ## Begin - Security
    # deny all direct access for these folders
    location ~* /(.git|cache|bin|logs|backups|tests)/.*$ {
        return 403;
    }

    # deny running scripts inside core system folders
    location ~* /(system|vendor)/.*\.(txt|xml|md|html|yaml|php|pl|py|cgi|twig|sh|bat)$ {
        return 403;
    }

    # deny running scripts inside user folder
    location ~* /user/.*\.(txt|md|yaml|php|pl|py|cgi|twig|sh|bat)$ {
        return 403;
    }

    # deny access to specific files in the root folder
    location ~ /(LICENSE|LICENSE.txt|composer.lock|composer.json|nginx.conf|web.config|htaccess.txt|$
        return 403;
    }
    ## End - Security

    ## Begin - PHP
    location ~ \.php$ {
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass unix:/var/run/php-fpm-yuumee.sock;
        # fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        #fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;
        fastcgi_intercept_errors off;
        fastcgi_buffer_size 16k;
        fastcgi_buffers 4 16k;
    }
    ## End - PHP

    gzip on;
    gzip_http_version 1.1;
    gzip_vary on;
    gzip_comp_level 6;
    gzip_proxied any;
    gzip_types application/atom+xml
               application/javascript
               application/json
               application/vnd.ms-fontobject
               application/x-font-ttf
               application/x-web-app-manifest+json
               application/xhtml+xml
               application/xml
               font/opentype
               image/svg+xml
               image/x-icon
               text/css
               text/plain
               text/xml;
    gzip_buffers 16 8k;
    gzip_disable "MSIE [1-6]\.(?!.*SV1)";

}
#5

Mind to have a look at the content of my NGINX config file I posted above? Something is definitely not right since I hit every URL.

#6

I don’t know NGIX, unfortunately. Blind child in the myst, here. :slight_smile: Maybe try without all those redirects first? I mean, get rid of HTTP:80 redirecting to HTTPS:443 and www.domainname redirecting to domainname, but I guess this is standard pattern in NGIX? You can also remove all PHP-related items (I assume you don’t use it on this site).

#7

I am still getting the issue. My SEO is really affected because of it. Anyone had similar issues? :frowning:

#8

If you still struggle about SEO try to block this content from google robots in robots txt or try to add meta robots noindex nofollow