Chief Delphi

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)
-   Python (http://www.chiefdelphi.com/forums/forumdisplay.php?f=187)
-   -   Progressively longer to initialize Python (http://www.chiefdelphi.com/forums/showthread.php?t=145628)

Arhowk 12-03-2016 21:48
Progressively longer to initialize Python
 
So we just came back from an event and I noticed as time went on, the starting of robot code took an increasing amount of time, starting from about 4 seconds at unbag to 15 seconds at bag. It didn't seem to be user code because I have prints at the start and the end of robotInit and it was printing the start message a second or two before the robot code led turns green. I was wondering if anyone else noticed this or has a reasonable explanation for the lag.

Also, kinda related, but halfway through the competition the first deploy stopped working. We had to cancel the first deploy while it was looking for the RIO and restart deploy to get it to work (sometimes multiple times). It wasn't connection or anything because I made sure the drive station showed Robot code led green before deploying but it was no help..

virtuald 12-03-2016 22:32
Re: Progressively longer to initialize Python
 
Quote:
Originally Posted by Arhowk (Post 1555997)
So we just came back from an event and I noticed as time went on, the starting of robot code took an increasing amount of time, starting from about 4 seconds at unbag to 15 seconds at bag. It didn't seem to be user code because I have prints at the start and the end of robotInit and it was printing the start message a second or two before the robot code led turns green. I was wondering if anyone else noticed this or has a reasonable explanation for the lag.
Did not have this problem today. Do you have gyros, camera, navx (though, we have a NavX and it works great).

Quote:
Also, kinda related, but halfway through the competition the first deploy stopped working. We had to cancel the first deploy while it was looking for the RIO and restart deploy to get it to work (sometimes multiple times). It wasn't connection or anything because I made sure the drive station showed Robot code led green before deploying but it was no help..
I think it's mdns related. I had issues with this yesterday on my mac, and basically each time that pyfrc executes a command (I think there's... 4 of them?), it redoes the mdns lookup, so each command is slloooooooow. I ended up just telling it to ssh to the self-assigned IP that the roborio gives itself, and then it was back to normal speed.

Arhowk 13-03-2016 12:56
Re: Progressively longer to initialize Python
 
Quote:
Originally Posted by virtuald (Post 1556016)
Did not have this problem today. Do you have gyros, camera, navx (though, we have a NavX and it works great).
We have a gyro and an IP camera but we've had both since day 1


Quote:
I think it's mdns related. I had issues with this yesterday on my mac, and basically each time that pyfrc executes a command (I think there's... 4 of them?), it redoes the mdns lookup, so each command is slloooooooow. I ended up just telling it to ssh to the self-assigned IP that the roborio gives itself, and then it was back to normal speed.
probably. After this event I absolutely hate mDNS. Switching to static IP at unbag time

virtuald 13-03-2016 22:51
Re: Progressively longer to initialize Python
 
Still have issues today? I would be interested in finding out more about it. No issues here.

Arhowk 14-03-2016 22:09
Re: Progressively longer to initialize Python
 
Heh, I'm afraid that the team will no longer be using Python after last weekend's competition (not my decision). Hopefully (not hopefully?) another team has this same issue and will be able to shed some light on it.

Arhowk 17-04-2016 12:43
Re: Progressively longer to initialize Python
 
So we switched back to Python and this time it didnt seem to get progressively longer, but it just seemed to take a generally long time. When we were using Java there was maybe 1-2 seconds between the robot code LED turning on and off and with Python there is a good 10 seconds. Any thought?

virtuald 17-04-2016 13:18
Re: Progressively longer to initialize Python
 
Quote:
Originally Posted by Arhowk (Post 1574249)
So we switched back to Python and this time it didnt seem to get progressively longer, but it just seemed to take a generally long time. When we were using Java there was maybe 1-2 seconds between the robot code LED turning on and off and with Python there is a good 10 seconds. Any thought?
My expectation is about 5-10 seconds. I haven't looked too deeply into this, but I believe it's because there's a lot of wpilib .py files that all have to be loaded, regardless of whether you use them or not. Loading the HAL ctypes wrappers is a lot of annoying overhead too.

Just did this experiment:

Code:
# time python3 -c 'import wpilib'

real        0m4.450s
user        0m3.410s
sys        0m0.430s

# time python3 -c 'import hal'

real        0m2.413s
user        0m1.250s
sys        0m0.210s
I would love to see this time go down, but I suspect it would take a lot of work to do so (one idea: cythonize everything). Pull requests welcome!

Arhowk 17-04-2016 13:40
Re: Progressively longer to initialize Python
 
Is it possible to preload these libraries outside of user code? It wouldn't help rio boot up time at all but it would minimize time between deploys

virtuald 17-04-2016 13:45
Re: Progressively longer to initialize Python
 
Python does support the reload() builtin, so in theory one could auto-reload code on demand. pyfrc's deploy even has an --inplace option that doesn't delete the existing code, so that the yeti framework can do this sort of thing. Would require some deep magic integration somewhere to make it more general.

Most of the time, the real answer is use ntproperty so that you have easy to use tunables, instead of redeploying code. :)

Arhowk 17-04-2016 13:50
Re: Progressively longer to initialize Python
 
Quote:
Originally Posted by virtuald (Post 1574292)

Most of the time, the real answer is use ntproperty so that you have easy to use tunables, instead of redeploying code. :)
I'm never relying on nettables for anything again after Kettering..

If the pyfrc libraries were compiled before being uploaded to the RIO (if they aren't already), would that make it significantly faster?

virtuald 17-04-2016 13:56
Re: Progressively longer to initialize Python
 
Quote:
Originally Posted by Arhowk (Post 1574293)
I'm never relying on nettables for anything again after Kettering..
As long as it connects (which we fixed by switching to static IPs), neither of my teams have had NetworkTables issues on the field that weren't of their own doing.

Quote:
If the pyfrc libraries were compiled before being uploaded to the RIO (if they aren't already), would that make it significantly faster?
Go look at /usr/local/lib/python3.5/site-packages/wpilib/__pycache__

vScourge 18-04-2016 09:46
Re: Progressively longer to initialize Python
 
Quote:
Originally Posted by Arhowk (Post 1574293)
If the pyfrc libraries were compiled before being uploaded to the RIO (if they aren't already), would that make it significantly faster?
Bytecode-compiled .pyc files are primarily to reduce file size, and only help where disk I/O is the bottleneck. If that's the case here then it may help but I wouldn't expect the difference to be dramatic.

Anecdotally, our robotpy deploy times dropped by at least a couple seconds once we switched our RoboRio to static IP and used that IP (not the hostname) to deploy. I would also second Dustin's advice on using tunables when practical, and just avoid redeploying for every tweak.

Although coming from using LabView in previous years our team still finds the robotpy deploy times to be a vast improvement.


All times are GMT -5. The time now is 21:37.

Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi