Keeping a Server alive

Technical Website Design/Showcase
1

Hey guys,
My RPi Web server always crashes at the wrong times (when I am not at home). I came up with this solution to reset it automatically:

Here is the sequence:
i2c interface between MCU and Pi
MCU controlling power to Pi via relay
MCU keeps pinging the Pi via i2C (sending a packet and waiting for response)
If the MCU fails the ping 5 times in a row, calls the reboot command (sudo reboot)
MCU waits 2 minutes
MCU pings the Pi a few times to see if it is alive again
Otherwise, MCU resets Pi by unplugging it and plugging it back in (relay)
MCU calls command to clear our corruptions on the HDD
MCU reboots server
Server up again!

2

Hmm. A better idea might be to look at why your web server crashes and fix it. Have you checked your server logs?

3

Well, yep. That is always the best idea. However, when running on a Raspberry Pi, getting DoSsed can easily crash the server!

4

What happens if your MCU crashes?

5

Maybe. But is that your problem? What are you using to serve your website? (Apache, Nginx, etc) Is it a problem with traffic to the website, or some other unhandled error on the server?

6

The typical DoS is high traffic. I have both Nginx and Apache installed. Apache handles dynamic HTML and Nginx handles the static files, wherever possible.

Running AB through localhost, I have tested my server at 100,000 hits, ten at a time! However, that was my nginx, which I have not released the port number to the public. WordPrss is a tad too heavy on the resources to allow more than 5 active connections at at time.

Also, is it often that MCUs crash? I have never heard of an Arduino crashing, because of how simple the bootloader and kernel is!

7

To echo a previous poster, find out exactly why the server is crashing. Is it too much memory usage? When it crashes, does the pi freeze, or does the service running the web server stop? You could make a simple script that restarts your web server once the memory usage exceeds 95%. Also, you could use cron to hourly restart the server if it has crashed.

8

What I mean by a crash is a kernel panic. The Pi is frozen. I cannot SSH to it. I cannot UART to it and I cannot connect a display, keyboard and mouse!
Before, server crashes were common. Recently, I have configured the server to refuse requests if it recieves too many. That way, DoSers are denied access to a 500 error message. Now, I would like the 500 message be sent to nginx instead, where less RAM will be used.

9

What model is it? Model A has problems from what i read on trhis site: http://www.raspberrypi.org/phpBB3/viewtopic.php?t=33763&p=319358

might be related to the Pi itself, and not just because its set as a webserver.:smiley:

10

I have a Model B, 512 MB. I know it isn’t the Pi’s fault because it is very stable until I send 100,000 requests in five minutes. The stability is mostly because the Pi runs Raspbian rom a hard disk instead of an SD/eMMC card!

11

This is fairly common and is called a watchdog. They make dedicated controllers or even circuitry in micro controllers to do this automatically.

12

Yes. I want to make it more-or-less a watchdog. Also, I would like that MCU controlling the cooling system because I currently have a fan that runs 24/7! That’s a lot of noise and wasted energy! I would also like to be able to log the boot and command input/output to a text file on an SD card to allow me to debug any crashes or kernel panics, if any text is sent over UART.