Server Restored
 

And, we're back.

I don't think there was any data loss .. let me know if you find anything weird happening by responding to this thread.

I haven't tuned mysql/apache completely, but will do so before things get too hectic.

Re: Server Restored
 

Wooooo! Yay for the OP!

Re: Server Restored
 

Yay! I can waste time at school again!

Congrats on getting the server back up!

Re: Server Restored
 

Brandon, Thanks.

Re: Server Restored
 

Thanks Brandon! Glad to have it back up. You realize quickly how much you are on Chief Delphi when you don't have it.

Re: Server Restored
 

This is probably a great opportunity to say thanks to you, Brandon. Not only for getting CD back up so promptly, but for all the work you've done over the years and continue to do for FIRST, VEX, and the STEM education community as a whole.

Thank you, sir.

Re: Server Restored
 

Day 3...after the downtime, I believe I have survived the withdrawal..

Re: Server Restored
 

OP delivered.

Thanks!

Re: Server Restored
 

At least it was November not January 10th :eek:

Great job!

Re: Server Restored
 

Thanks to Brandon for your continued hard work with the site. It is truly appreciated!

Re: Server Restored
 

Thanks for being so quick about it. Love the site.

Re: Server Restored
 

I survived solely on 610's facebook posts/youtube videos these past few days.


Thanks for a speedy restoration & all the time you put in!

Re: Server Restored
 

YAY! THANKS!!!

Re: Server Restored
 

Great to have the site up and running, thanks for the hard work you put in for the site.

Re: Server Restored
 

What happened anyways, was it just a server issue? Though thankgod it is back.

Re: Server Restored
 

Thanks for getting the forum back up quickly!

Out of curiosity, where/how is the website hosted, and what sort of server is it?

Re: Server Restored
 

The server was on SoftLayer's CloudLayer system. There was some data corruption (they say), the server crashed and wouldn't boot back up.

I ended up moving everything over to RackSpace's Cloud Servers, since I'm more familiar with their setup (use them at work and for other projects) and it's a little bit cheaper for the computing power you get, I believe.

We're currently sitting on a cloud server with 15GB RAM and 6 vCPUs.

Re: Server Restored
 

Same Here! LOL!

Re: Server Restored
 

So, was there RAID? If there was, what a coincident to have all the disks fail at the same time! I think this culd be partially caused by bad server maintenance! Also, do you use SSH to fix things or do you go to the hosting provider? For me, I do both (Server located in my room!)

Re: Server Restored
 

And spam is back up again too. Sigh. :( :(

Re: Server Restored
 

I think the clock is off a bit -- CD sent me the automatic birthday greetings this evening at 7:01 pm local time, and there's a birthday cake next to my name, but it's not until tomorrow the 13th. (And the "Today's Birthdays" list doesn't have my name... yet?) What time zone does the server think it's in???

And thanks, Brandon, for the restoration!

Re: Server Restored
 

Yep, timezone was set incorrectly (UTC).

Should be fixed now.

Re: Server Restored
 

This is really a minor detail, but the stats page at http://www.chiefdelphi.com/stats/
is missing past march 2012.

Re: Server Restored
 

Getting closer! Got another b'day email at 8:18 pm. :) (But lost the cake :( ) I like this; can you just keep jiggling the clock every hour or so, so I keep getting these emails?

Re: Server Restored
 

Yeah, that's me just being lazy. :)

Re: Server Restored
 

I literally had to look at my entire life while it was down. SO glad its back!

Re: Server Restored
 

+1

Thank you Brandon!

Re: Server Restored
 

Yeah, also, thank you for giving me some time for homework! I owe you a big one! :D :D :D :D :D

Re: Server Restored
 

Not sure if this was planned/preexisting or not... but a number of the gray box links near CD-Media and CD-Spy have disappeared (like the direct link for UFH--though that might have disappeared earlier and I just didn't notice).

Re: Server Restored
 

Some of them have been gone for a while, as part of the changes mentioned in this thread. I removed UFH last night when I was looking over things -- all of the previous UFH honorees profiles have been moved to their discussion thread in the UFH sub-forum.

Re: Server Restored
 

This isn't a new bug, but it isn't possible to sponsor through paypal. It reports no subscriptions available.

Re: Server Restored
 

Yea when the site went down I was hoping they were making improvements to the spam filter. Well O well, I hope they get it fixed by the time of FRC 2014 kickoff or who know the person spamming will increase their spamming.

Re: Server Restored
 

I saw a small amount of time on 11/13 at around 10:11PM that the server was down, even checked with isitdownrightnow.com and downforeveryoneorjustme.com and they both saw it down as well. Was this just a small update? or was there another failure of some sort? or was it just me?

Re: Server Restored
 

Nothing went on that I know of .. the monitoring I was using was every 30 min. I just switched over to a new monitor to check every 5 min., in case it happens again.

Re: Server Restored
 

Like i said, most likely my own user error, though I do wonder why those two sites saw it as "down" as well :eek:

Re: Server Restored
 

This is off topic, but do you have any backup service like CloudFlare? What about a backup server, or even a node system, where users can install an app on their computer which will allow their computers to be a server (very low load), especially during peak times?

Re: Server Restored
 

Maybe the internet connection just had a "hiccup." This means that the connection will be lost for a few seconds to a few minutes and then reconnect.

I saw a backup occur at 11 est, rather then later. Seems like another Utc bug.

Re: Server Restored
 

I may not have restarted Apache after fixing the timezone.

If I remember, I'll check tonight at 11 to see if it's still happening. (Should be happening at 4am eastern).

Re: Server Restored
 

There was a backup at 12 today. You may want to change that to a weekly or monthly backup instead because for us, in the west coast, this is the time that we finish our homework and actually get active online

Re: Server Restored
 

This should happen at 4am Eastern .. I've made another change to the schedule.

Let me know if it comes up again tonight.

Re: Server Restored
 

I wonder who surfs at 4AM? Also, why do you go by daily backups? I back up my website every three months. Also, if anything seems to be brewing, I have the ability to switch hard drives and move the data flawlessly.

As a matter of fact, I haven't backed up my server yet! I should maybe have a shadow copy of the database in the eMMC, but it doesn't really matter because the hard drive is only an year old

Re: Server Restored
 

4AM eastern is 1AM pacific. I know that I am sometimes on at midnight. Also I would think CD is used quite a bit more frequently than your personal website.

Re: Server Restored
 

I agree with that. That is why I don't like the idea of having daily backups, where the entire site is shut down!

Re: Server Restored
 

The backups are quick and if it is what is needed to potentially save a month worth of data then it is completely with it IMO.

Re: Server Restored
 

Yes, the backups are pretty fast. However, this is a big site, so I believe it takes up to an hour normally, which can get a little annoying!

Re: Server Restored
 

For some reason I remember them being max 15 minutes long(I could be wrong though so please correct me if I am). You only have to back up new information, so I can't fathom it being that long. Very few people are on from 1-4AM anyways(except for the people in Israel), so I think most people prefer the quick backup every day that almost always doesn't affect them over a much longer backup every month that would likely bring the sight down for a significant amount of time. The daily 15 min backups would turn into a 7.5 hr backup.

Re: Server Restored
 

Sometimes I'm still up at 4AM eastern. :rolleyes:

Re: Server Restored
 

Yeah, my "Bed time" is when CD starts to back up.

Re: Server Restored
 

I've been noticing a long delay (10s of seconds) when reporting a post. Has anyone else seen that?

Re: Server Restored
 

Yeah, I've noticed it too. If I'm not mistaken, was reporting a post previously considered making a post? Because I'd always get the "one post in 47 seconds" message, up until recently, which I'm guessing that change came with the delay...

Re: Server Restored
 

I thought that was just a bug. Now I think that the mail server could be acting up and causing a delay

Re: Server Restored
 

Yes, It's been like this for a long time. I sometimes worry if it goes through, especially when reporting from my phone.

Re: Server Restored
 

I was just able to report a post and it took 1-2 seconds to do .. not sure what would cause the delay you're seeing.


Also -- I've added some additional IP filtering. Any IP originating in certain countries will be blocked, and given instructions on how to unblock themselves (e-mail me). I have initially blocked out the countries with IPs that have been causing a lot of the spam recently (India and Bangladesh).

Re: Server Restored
 

Doesn't that mean you'll be getting hundreds of spam emails from India and Bangladesh?

Re: Server Restored
 

Most likely the spam bots will just move onto another vBulletin site.

Re: Server Restored
 

It just worked for me in about 3 seconds, much faster then before.

Re: Server Restored
 

Next time it takes 10sec+, send me an email with the approximate time, so I can look through some monitoring I have set up.

Re: Server Restored
 

It may be related to the number of moderators it is sending an email to. I just reported a thread in general forum (with 35 moderators) at 1:50 PST, and it took almost 30 seconds. Our previous tests would have only sent emails to a few moderators.

Re: Server Restored
 

It happened to me just 30 minutes ago, at 4:22 when reporting this thread http://www.chiefdelphi.com/forums/sh...93#post1317493

I explained my concerns and clicked Submit, then waited maybe 10 seconds before it responded.

Re: Server Restored
 

That SFS Plugin should prevent that, not only on vBulletin, but on other platforms like SMF too.

Re: Server Restored
 

I just experienced this problem (just to make aware that this is still an issue of some sorts) reporting a post in the general forum at around 11:01 PM EST. Wrote about a sentence in the explanation box, then took at least ten seconds for the report to go through

Re: Server Restored
 

It appears that awstats.pl is no longer executable, and that stats.chiefdelphi.com no longer exists, so it's more then just not adding links

Re: Server Restored
 

Is is just me, or is this site slowing to a crawl every time someone presses the [Submit Reply] button?

Re: Server Restored
 

The website works well for me.