RPGWatch - Moving to a new server completed
- Thread starter Myrthos
- Start date
Most are probably being directed from the RPGWatch Steam Curator page. As the last time the site had high number was when Obama won his first Presidency in 2009.
Correction you might be getting doxxed.
Cacheperl
SasqWatch
- Joined
- May 18, 2012
- Messages
- 2,315
Correction you might be getting doxxed.
ddos, not doxxed.
If it was the steam thing, it should be visible in the statistics about referring sites?
- Joined
- May 18, 2012
- Messages
- 2,315
Well took me ten minutes just to load this thread and reply.
You're right I meant to say under DoS or DDoS attack.
You're right I meant to say under DoS or DDoS attack.
I am curious what the root problem was if you don't mind sharing. Also the site has been very slow over weekend. Not sure if this related to the new software/site or DOS attack.
-
Btw is your web-sever event based (epoll) or do you use apache (select/poll) (which supports limited number of connections and generally forks).
-
Btw is your web-sever event based (epoll) or do you use apache (select/poll) (which supports limited number of connections and generally forks).
Carnifex
SasqWatch
The site has seemed a bit slower to me over the past two days, but I wrote it off to the work that was being done.
S
sakichop
Guest
Sites very slow for me as well. Also when I click on the Rpgwatch logo at the top left of the screen to go to the home page it always sends me to the time wanderer Kickstarter. Then I have to refresh the page and the home page will load.
The root problem was several Chinese IPs that were continuously making connections in parallel to RPGWatch. Yesterday I figured that out and blocked the IP ranges, but it didn't help a lot. So I looked into other reasons. So I ended up balancing the memory needs for the database and Apache, to allow the site to remain alive, albeit slow.
Last night at 3 (CET), it looked OK.
Today we still had issues, so I went through the log files again, to find the same IP addresses that were apparently not blocked. It took me quite some time to find out that the addresses are only blocked when I reboot Apache, which is very odd.
After that the number of visitors dropped rapidly to normal numbers.
Last night at 3 (CET), it looked OK.
Today we still had issues, so I went through the log files again, to find the same IP addresses that were apparently not blocked. It took me quite some time to find out that the addresses are only blocked when I reboot Apache, which is very odd.
After that the number of visitors dropped rapidly to normal numbers.
- Joined
- Aug 30, 2006
- Messages
- 11,223
I appologize I mean root cause for the quoting issue. Yea - there are a lot of sites that run robots that poll the ip space. The problem is that if you are sufficiently slow the connections will begin to backup until things fall over. I think there is something else going on here as it seems that initial connections are rather heavy weight on your sever - though apache itself is not great.
An initial connection shouldn't require db hit (we hope); not sure if your setup caches the front page or if every connection generates a new front page from the db (which would be expensive; esp for a robot hit). What is worse is some of these robots will try to log in using a dictionary of names/pwd - waste really but since it doesn't cost them anything to run a robot....
An initial connection shouldn't require db hit (we hope); not sure if your setup caches the front page or if every connection generates a new front page from the db (which would be expensive; esp for a robot hit). What is worse is some of these robots will try to log in using a dictionary of names/pwd - waste really but since it doesn't cost them anything to run a robot....
- Joined
- April 12, 2009
- Messages
- 23,459
Chinese haxxorz?
The site still has very slow response occassionally on my side. Don't get me wrong but it's fine by me actually - some other sites are slower than this.
The site still has very slow response occassionally on my side. Don't get me wrong but it's fine by me actually - some other sites are slower than this.
- Joined
- Apr 12, 2009
- Messages
- 23,459
Every https connection will result in some database queries, as it needs to verify the user and get the configuration of that user. As a result there is an SQL connection.
Pages are cached, so when nothing changes, you get a cached pages, but first will have to get past the SQL queries.
We are using InnoDB and it likes to have a fair amount of memory allocated, so it can keep stuff in memory. Then there is PHP-FPM that needs more memory with every connection. This requires balancing the two as we are running a server with 16GB, which is usually more than enough, but not in these cases. I also never optimized the queries for large amount of visitors as there never was any need, so this adds extra delays when visiting the front page.
This stuff happens every now and then and usually the site was not reachable. Now it at least loads, slowly, but still it loads, so I call that progress
The character issue is related to a difference in the database and PHP. On the old server we used PHP 5.x and MySQL. On this server we use PHP 7.1 and MariaDB.
Our database uses Latin1 as character set, which is also the default of PHP and MariaDB. At least that is what the documentation mentions. When a connection was made with the database from PHP a character set was not explicitly set. That worked on the old server, but not here. Apparently the default is not Latin1 anymore. So, I explicitly set the character set now to Latin1 when making a connection.
Never trust defaults…
Pages are cached, so when nothing changes, you get a cached pages, but first will have to get past the SQL queries.
We are using InnoDB and it likes to have a fair amount of memory allocated, so it can keep stuff in memory. Then there is PHP-FPM that needs more memory with every connection. This requires balancing the two as we are running a server with 16GB, which is usually more than enough, but not in these cases. I also never optimized the queries for large amount of visitors as there never was any need, so this adds extra delays when visiting the front page.
This stuff happens every now and then and usually the site was not reachable. Now it at least loads, slowly, but still it loads, so I call that progress
The character issue is related to a difference in the database and PHP. On the old server we used PHP 5.x and MySQL. On this server we use PHP 7.1 and MariaDB.
Our database uses Latin1 as character set, which is also the default of PHP and MariaDB. At least that is what the documentation mentions. When a connection was made with the database from PHP a character set was not explicitly set. That worked on the old server, but not here. Apparently the default is not Latin1 anymore. So, I explicitly set the character set now to Latin1 when making a connection.
Never trust defaults…
- Joined
- Aug 30, 2006
- Messages
- 11,223
Thanks! The latin1 issue was what i wanted to know...
i am confuse when a connection first comes in why does it have to query the db for the user if there is no user (I am assuming that these robot hits don't actually have a cookie identifying them as a logged in user but perhaps I am mistaken?). If this the case could a short cut be made for connections that don't carry the appropriate cookie to bypass the database hit ?
i am confuse when a connection first comes in why does it have to query the db for the user if there is no user (I am assuming that these robot hits don't actually have a cookie identifying them as a logged in user but perhaps I am mistaken?). If this the case could a short cut be made for connections that don't carry the appropriate cookie to bypass the database hit ?
Every https connection will result in some database queries, as it needs to verify the user and get the configuration of that user. As a result there is an SQL connection.
Pages are cached, so when nothing changes, you get a cached pages, but first will have to get past the SQL queries.
We are using InnoDB and it likes to have a fair amount of memory allocated, so it can keep stuff in memory. Then there is PHP-FPM that needs more memory with every connection. This requires balancing the two as we are running a server with 16GB, which is usually more than enough, but not in these cases. I also never optimized the queries for large amount of visitors as there never was any need, so this adds extra delays when visiting the front page.
This stuff happens every now and then and usually the site was not reachable. Now it at least loads, slowly, but still it loads, so I call that progress
The character issue is related to a difference in the database and PHP. On the old server we used PHP 5.x and MySQL. On this server we use PHP 7.1 and MariaDB.
Our database uses Latin1 as character set, which is also the default of PHP and MariaDB. At least that is what the documentation mentions. When a connection was made with the database from PHP a character set was not explicitly set. That worked on the old server, but not here. Apparently the default is not Latin1 anymore. So, I explicitly set the character set now to Latin1 when making a connection.
Never trust defaults…
Its very slow for me as well.
Yep it takes 3-10 minutes for me again.
Basically to many guests over 3,000 again.
Basically to many guests over 3,000 again.
Record number of users today at over 5K
Last edited: