Control FTC4314's robot over the web
 

We have created an applet to control our teams robot over the web.
the url is here: http://team0x27.org/robot/

please be courteous and do not stall the motors or drive into anything.

the robot may or may not be available at all times.
If you cannot connect it means somebody else is connected at the moment so you can keep trying to connect.

I will try to get a video stream up so you can watch the robot even if you are not driving it.

Re: Control FTC4314's robot over the web
 

After a few people driving it, it seems like we need to find a more foolproof environment for the robot. many things that can go wrong in the current locations. Will update soon.

Re: Control FTC4314's robot over the web
 

The link ^ doesnt work..

Re: Control FTC4314's robot over the web
 

This is a very, very cool idea. Let us know when it gets back up, I wanna play. :)

Re: Control FTC4314's robot over the web
 

:D

I have a friend who did something similar with the basement of his house. You can turn off the lights, move a robot arm, and several other things through the internet.

I wanna play too!

Re: Control FTC4314's robot over the web
 

There is no possible way this could go wrong in the hands of someone with poor driving skills.

Me next!

Re: Control FTC4314's robot over the web
 

Just improved to framerate from about 2 frames per second to ~15
The plan for now is to create an admin piece of is to create an admin piece of software that can override the users commands.

Then make it available to public at some times when someone can supervise the robot.

Re: Control FTC4314's robot over the web
 

I got the chance to drive it a little, and man was it hard! The Lag from video to actual moment was pretty great and it was not hard to run into things. ::safety::

One thing I thought was pretty cool was the little arm you had on the front of the robot, to move the ramp down to go up it. Now that was pretty cool!

All in all, it was fun and I would love to drive it again. Plus it could be a BIG hit for people that are bored and looking for something to online.

Re: Control FTC4314's robot over the web
 

I wanna drive!

I wonder if we could do this with our FRC robot. Just have to make sure we don't put holes in the walls or anything :yikes:

Re: Control FTC4314's robot over the web
 

Link to drive will not always be public. Too risky in terms of stall,burnout and damage to bot and its environment to leave it running. You can however watch on ustream and see team members driving via the web.

http://www.ustream.tv/channel/live-robot

From time to time we will make it available for driving by the public. The link will be posted in the ustream chat.

You can also look at the status of the Samantha system eg. batery voltages for the NXT as well as the main 12V battery... IP address of who is in control of the bot and a bunch of other stuff here.

Will post this address after we evaluate the security/damage risk.

On board video will be the next step.

FYI: ustream is delayed by 3+ seconds and is almost useless for control unless you are thinking hybrid mode like FRC 2008.

The video latency and frame rate on the driving applet was just improved by moving from grabbing individual jpegs to streaming motion jpeg. The camera is and axis207W. Very similar to the AXIS cam used for FRC.

THANKS

Thanks to John Toebes (creator of the FTC Samantha wireless control system) for providing the challenge of doing this... as well as providing information on protocol. So far we are really happy with the SAmantha system. We are FTC Rookies and the Samantha is apparently well received by other teams who suffered difficulties with the previous Bluetooth system.

HOW WE DID THIS

We had been thinking about this for a couple years now in terms of a web app for the general public. We had actually driven the FRC bot via wireless using the camera during the 2009 controls system beta test in partnership with team 842. It was basically the very first thing we did when we got the 2008 bot driving with the cRio. We never did get it running on the web though.

Then last Wed.on the Samantha Conference call with John Toebes I asked about getting access to the protocol for this purpose and John mentioned that some teams had talked about doing this but nothing had been accomplished yet. He agreed to provide some info on protocol. To keep it simple, we did not want to do a custom app on the bot but rather just have our over the WEB control system masquerade as an FCS.

I was not however able to contact John until Friday night but we (mostly Borna) started coding the Java applet Thursday night. On Saturday, using Wireshark we sniffed out the essence of the protocol in an hour and got that going (Borna remotely via screen sharing on skype) but could not sort out the initialization (related to the pairing process) that is used as part of the "security" to prevent unauthorized IP address form controlling the Samantha. FYI, the Samantha is WiFi. After a discussion with John on Saturday, he agreed to cut a version of the Samantha code that could be initialized in a mode that does not use the pairing and this allowed the Java applet that we had developed to run with almost no modification. We were driving over the WEB by midnight Saturday. Then we embedded the video into the JAva applet to make it even cleaner. Borna and Austin have been tweaking and polishing it since then.

One of the more difficult things was figuring out how to do a security certificate for the Java app so that it will run over the WEB. No problem developing on a LAN but as soon as the browser sees that it is on the open WEB, security precautions go up by some 30 dB.

We will get a writeup on either the Team 39 and/or the Team0x27 (NURC) web site.

Regards
Frank
Mentor
FRC Team 39
FTC 4314
NURC 0x27

Re: Control FTC4314's robot over the web
 

Its down, but how can you regulate how many people control it at one time? Do you even regulate that? What if we run over important stuff, or ram into the wall several time?

Re: Control FTC4314's robot over the web
 

The applet that runs on your PC to give you the viewing window and the control buttons is part of the process of preventing additional connect requests to the bot if someone is already connected. The java applet on your machine checks the Samantha using http: (port 80) to see if anyone is connected to the Samantha already. This is the same status html page that has the battery voltages etc. If no one is connected yet, the applet on your browser continues and connects you to the robot. There may be a possible race condition in terms of second user attempting to connect but the ultimate back up mechanism that prevents more than one connection from being accepted is that the Samantha only allows one TCP connection on the control port (not the status port) anyway.

Will eventually do some sort of driver queuing system but it needs a server somewhere to act as the central memory keeping track of requests.... checking the blacklist etc. Probably one of the PC's here with the bot.

An update in the next couple days will put status info like battery voltage and current drivers IP address on the applet for people to watch as they are waiting to drive. It updates at ~ 1 Hz.

One of our hopes in doing this is that FRC or FTC will consider a Tele-present mode in addition to Autonomous and Tele-operated modes.

Regards
Frank
FTC 4314
FRC 39
NURC 0x27

Re: Control FTC4314's robot over the web
 

The robot is back online for a while.

http://team0x27.org/robot/

Re: Control FTC4314's robot over the web
 

It has no real purpose, I would call it novelty LOL. But congratulations to you guys

Re: Control FTC4314's robot over the web
 

Amazing. Sorry I drove it off the side of the ramp :ahh:

Re: Control FTC4314's robot over the web
 

nerds!

:)

Fun stuff, I got to drive for a little bit. I went forwards and backwards. That's how I figured out which way the robot was pointing.

Re: Control FTC4314's robot over the web
 

davidthefat,

The purpose of the exercise was ...

1 To learn the details needed to implement minimal robot telepresence via a WEB app.

2. To benchmark and analyze performance limitations of our approach adn change that approach over a couple of iterations of improvements.

3. To possibly inspire interest at FTC and FRC in terms of a telepresent mode during game play.

4. Share the knowledge so other teams can have an easy starting point in doing this themselves.

5. To just do it and have some fun having people from half way across the country or even the world amaze themselves with "action at a distance....".


Some times it was real slick and at other timse it was trying to have telepresence with a bot on the moon and the several second latency on the video.

So we are done public testing for tonight and they are working on improvements already. We had over 120 people tune in via ustream tonight and had some 14 or so viewing at one time. No idea how many people drove.

Thanks everyone that helped.

Frank
FTC 4314
FRC 39
NURC 0x27

Re: Control FTC4314's robot over the web
 

Just a quick update for when the applet come back online later...


- Added CPU friendly mode toggle (lowers applet FPS)
- Added real time FPS counter
- Changed applet layout. All buttons now appear alongside the camera feed, hopefully now more people can see the full image and controls at the same time. Also moved Left/Right buttons closer together.
- Removed user checking*
- Added WASD key controls
- Added Arrow key controls


* To anyone who had trouble getting the robot to move yesterday please try it again as this hopefully fixed the main problem. The way it was set up people idling in the app were sending html requests to check the active connection IP to the Samantha 2 times per second... If you have 10 people with the applet open that's one request every 50ms! The poor little Samantha controller buckled under the pressure. Ah well, lesson learned. One more reason to start work on some intermediary server software. :P

Hopefully the new version will be online sometime soon later today. Testing at 2AM I was able to get a solid 30fps from the camera. :ahh: Hopefully it functions at least half as well with users farther away.

Also, what kind of features would the CD community find useful or interesting in an program like this?

Regards,

- Austin

Re: Control FTC4314's robot over the web
 

Let us know when it goes live again!

Also a suggestion, maybe you could have the robot time out after 5 minutes or so, that way one person can't control the robot for hours at a time.

Cool project.

-Brando

Re: Control FTC4314's robot over the web
 

^ This. Put a countdown on the applet to let users know they have a time limit. I'm sure this will become more and more popular as the thread gains more views.

I can't wait to try it out!

Re: Control FTC4314's robot over the web
 

the applet already has time restrictions. each person is disconnected after 120 seconds(the countdown is displayed on the applet). Also if they have been idle for more that 15 seconds they are disconnected.

I will create a page that shows what the applet looks like when we have it offline so the link on the first post loads correctly.

Re: Control FTC4314's robot over the web
 

Does anyone have an idea on how to notify people when the robot goes live? I will post on here but that would not reach many of the users.

Re: Control FTC4314's robot over the web
 

Some sort of Twitter account that people can follow. If it doesn't go up-and-down often, a Team's existing account wouldn't be a bad idea. Otherwise, Twitter is free, so a dedicated account wouldn't take much effort to create.

Maybe e-mail notifications. Create a page where people can sign up to receive an e-mail when the system goes up.

Re: Control FTC4314's robot over the web
 

We also put video on ustream so that if you are not driving you can watch. In the ustream chat we will notify ~ when we will be putting the bot on line.
the ustream feed is at http://www.ustream.tv/channel/live-robot

There are too many risks so we can just leave the Telepresence demo unsupervised. As we are building today there may be long periods of what looks like nothing happening with people working in the garage.

Tentatively looking at 1 pm AZ time (GMT- 7) today to put the bot on line for say 30 minutes of driving.

If you get to drive, please just blip the controls. Holding the buttons down will just ram the bot into stuff and possibly stall/damage motors. If we see this, we will take the bot off line. So please drive it like you are driving the last surviving Mars rover. The robot con only get on the tippy ramp going forward. This is because the right front of the bot has a deflector to tip the ramp downwards as the bot approaches the ramp. So try to approach the ram straight on. no driving on and off the sides of the ramp.

Summary: Don't drive this like a 4x4 off road RC truck. Drive it like a Mars rover. i.e. blip the control...then wait for something to happen. Blip again....etc. Don't hold button down for more than 1/2 sec and things will stay safe. Please do not chase the pets.

Frank
FTC 4314
FRC 39
NURC 0x27