Go to Post Hundreds of teams. Thousands of students. Countless memories. Infinite fun. You'll love it, I promise you that. - smurfgirl [more]
Home
Go Back   Chief Delphi > Technical > Programming
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Closed Thread
Thread Tools Rate Thread Display Modes
  #1   Spotlight this post!  
Unread 18-12-2013, 23:14
Thad House Thad House is offline
Volunteer, WPILib Contributor
no team (Waiting for 2021)
Team Role: Mentor
 
Join Date: Feb 2011
Rookie Year: 2010
Location: Thousand Oaks, California
Posts: 1,107
Thad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond repute
Fixing opencv lag

I am trying to get camera tracking working on my beaglebone black. I want to use the axis camera to do this, and I can get that part working. The problem is that the tracking is limited to about 15 frames per second, and the camera is sending at 30fps. This is causing images to just get stored in the buffer, which then starts lagging badly. Does anybody know how to either shrink the buffer down to 1 image, read from the back of the buffer, or loop through the buffer to the end quickly. Or if there are any other suggestions that would help too.

Also this is unrelated, but does anybody know why no networking programs allow reading from the back of the buffer. I have wanted to do that MANY times, but nothing ever supports it, and just curious if anybody knows why that is.
__________________
All statements made are my own and not the feelings of any of my affiliated teams.
Teams 1510 and 2898 - Student 2010-2012
Team 4488 - Mentor 2013-2016
Co-developer of RobotDotNet, a .NET port of the WPILib.
  #2   Spotlight this post!  
Unread 19-12-2013, 00:11
otherguy's Avatar
otherguy otherguy is offline
sparkE
AKA: James
FRC #2168 (The Aluminum Falcons)
Team Role: Mentor
 
Join Date: Feb 2010
Rookie Year: 2009
Location: CT
Posts: 443
otherguy is a splendid one to beholdotherguy is a splendid one to beholdotherguy is a splendid one to beholdotherguy is a splendid one to beholdotherguy is a splendid one to beholdotherguy is a splendid one to beholdotherguy is a splendid one to behold
Re: Fixing opencv lag

This is basically what I was going to write, but Zaphod said it better...
Quote:
...have a separate capture thread, in addition to your [image processing] thread. The capture thread keeps reading in frames from the buffer whenever a new frame comes in and stores it in a "recent frame" image object. When the [image processing] thread needs the most recent frame, it locks a mutex for thread safety, copies the most recent frame into another object and frees the mutex so that the capture thread continues reading in new frames.
http://stackoverflow.com/questions/1...-reading-issue
__________________
http://team2168.org
  #3   Spotlight this post!  
Unread 19-12-2013, 00:15
Thad House Thad House is offline
Volunteer, WPILib Contributor
no team (Waiting for 2021)
Team Role: Mentor
 
Join Date: Feb 2011
Rookie Year: 2010
Location: Thousand Oaks, California
Posts: 1,107
Thad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond repute
Re: Fixing opencv lag

Yeah I was just about to post that I read that. I did it without the locks because I don't know how to do locks in python, but thats coming up next.

Thanks
__________________
All statements made are my own and not the feelings of any of my affiliated teams.
Teams 1510 and 2898 - Student 2010-2012
Team 4488 - Mentor 2013-2016
Co-developer of RobotDotNet, a .NET port of the WPILib.
  #4   Spotlight this post!  
Unread 19-12-2013, 01:42
billbo911's Avatar
billbo911 billbo911 is offline
I prefer you give a perfect effort.
AKA: That's "Mr. Bill"
FRC #2073 (EagleForce)
Team Role: Mentor
 
Join Date: Mar 2005
Rookie Year: 2005
Location: Elk Grove, Ca.
Posts: 2,384
billbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond repute
Re: Fixing opencv lag

Quote:
Originally Posted by sst.thad View Post
I am trying to get camera tracking working on my beaglebone black. I want to use the axis camera to do this, and I can get that part working. The problem is that the tracking is limited to about 15 frames per second, and the camera is sending at 30fps. This is causing images to just get stored in the buffer, which then starts lagging badly. Does anybody know how to either shrink the buffer down to 1 image, read from the back of the buffer, or loop through the buffer to the end quickly. Or if there are any other suggestions that would help too.
Is there a reason you are not just requesting one frame and then processing it before requesting the next?
In other words, don't allow the camera to "free run".
You might also try setting the Axis to run at 15 fps. As long as you can process at 15 fps, you should not be filling the buffer.
__________________
CalGames 2009 Autonomous Champion Award winner
Sacramento 2010 Creativity in Design winner, Sacramento 2010 Quarter finalist
2011 Sacramento Finalist, 2011 Madtown Engineering Inspiration Award.
2012 Sacramento Semi-Finals, 2012 Sacramento Innovation in Control Award, 2012 SVR Judges Award.
2012 CalGames Autonomous Challenge Award winner ($$$).
2014 2X Rockwell Automation: Innovation in Control Award (CVR and SAC). Curie Division Gracious Professionalism Award.
2014 Capital City Classic Winner AND Runner Up. Madtown Throwdown: Runner up.
2015 Innovation in Control Award, Sacramento.
2016 Chezy Champs Finalist, 2016 MTTD Finalist
  #5   Spotlight this post!  
Unread 19-12-2013, 02:06
Thad House Thad House is offline
Volunteer, WPILib Contributor
no team (Waiting for 2021)
Team Role: Mentor
 
Join Date: Feb 2011
Rookie Year: 2010
Location: Thousand Oaks, California
Posts: 1,107
Thad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond repute
Re: Fixing opencv lag

I tried just requesting a single frame at a time, but it would take 100ms from requesting the image to actually receive the image. It was a little too much for me.

As for limiting the fps it wasnt working for some reason.
__________________
All statements made are my own and not the feelings of any of my affiliated teams.
Teams 1510 and 2898 - Student 2010-2012
Team 4488 - Mentor 2013-2016
Co-developer of RobotDotNet, a .NET port of the WPILib.
  #6   Spotlight this post!  
Unread 19-12-2013, 02:18
wireties's Avatar
wireties wireties is offline
Principal Engineer
AKA: Keith Buchanan
FRC #1296 (Full Metal Jackets)
Team Role: Mentor
 
Join Date: Jan 2006
Rookie Year: 2004
Location: Rockwall, TX
Posts: 1,171
wireties has a reputation beyond reputewireties has a reputation beyond reputewireties has a reputation beyond reputewireties has a reputation beyond reputewireties has a reputation beyond reputewireties has a reputation beyond reputewireties has a reputation beyond reputewireties has a reputation beyond reputewireties has a reputation beyond reputewireties has a reputation beyond reputewireties has a reputation beyond repute
Send a message via AIM to wireties
Re: Fixing opencv lag

Quote:
Originally Posted by sst.thad View Post
I tried just requesting a single frame at a time, but it would take 100ms from requesting the image to actually receive the image. It was a little too much for me.

As for limiting the fps it wasnt working for some reason.
Just toss every other frame? Have you tried using C or C++ instead of Python?
__________________
Fast, cheap or working - pick any two!
  #7   Spotlight this post!  
Unread 19-12-2013, 02:46
Thad House Thad House is offline
Volunteer, WPILib Contributor
no team (Waiting for 2021)
Team Role: Mentor
 
Join Date: Feb 2011
Rookie Year: 2010
Location: Thousand Oaks, California
Posts: 1,107
Thad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond repute
Re: Fixing opencv lag

Quote:
Originally Posted by wireties View Post
Just toss every other frame? Have you tried using C or C++ instead of Python?
I know the python libraries. And the threaded way works just fine, and most likely better.
__________________
All statements made are my own and not the feelings of any of my affiliated teams.
Teams 1510 and 2898 - Student 2010-2012
Team 4488 - Mentor 2013-2016
Co-developer of RobotDotNet, a .NET port of the WPILib.
  #8   Spotlight this post!  
Unread 19-12-2013, 05:49
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,756
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: Fixing opencv lag

The WPI libraries in LV include vision code. The code uses parallel loops -- results in parallel threads -- in case the user has set the camera faster than they can process. This is effective for keeping the lag to a minimum.

You may also want to lower the camera frame rate, though. First, there is some overhead involved in processing the frames, though hopefully you aren't decompressing the images or doing any of the expensive things. Second, the two thread approach will result in much less certainty about when the image was acquired. If you are trying to estimate time, it is likely better to run the camera at a slower, but known rate that doesn't buffer images.

The other approach of requesting an image only when you want one was used in older versions of WPILib. It worked/works far better on the 206 model than on the newer Axis cameras. There is indeed quite a bit of setup overhead and the mjpeg approach is almost required to get above 10fps.

Greg McKaskle
  #9   Spotlight this post!  
Unread 19-12-2013, 14:20
Thad House Thad House is offline
Volunteer, WPILib Contributor
no team (Waiting for 2021)
Team Role: Mentor
 
Join Date: Feb 2011
Rookie Year: 2010
Location: Thousand Oaks, California
Posts: 1,107
Thad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond repute
Re: Fixing opencv lag

Quote:
Originally Posted by Greg McKaskle View Post
The WPI libraries in LV include vision code. The code uses parallel loops -- results in parallel threads -- in case the user has set the camera faster than they can process. This is effective for keeping the lag to a minimum.

You may also want to lower the camera frame rate, though. First, there is some overhead involved in processing the frames, though hopefully you aren't decompressing the images or doing any of the expensive things. Second, the two thread approach will result in much less certainty about when the image was acquired. If you are trying to estimate time, it is likely better to run the camera at a slower, but known rate that doesn't buffer images.

The other approach of requesting an image only when you want one was used in older versions of WPILib. It worked/works far better on the 206 model than on the newer Axis cameras. There is indeed quite a bit of setup overhead and the mjpeg approach is almost required to get above 10fps.

Greg McKaskle
We are using a Beaglebone black, which makes us run openCV. If it could run labVIEW we would use it. For some reason limiting the FPS was not working.

I ran the threading overnight last night, and it ran for 12 hours without missing any frames or starting to lag. How I had it set up was grab the image in the fast looping thread, and store that. Then in the slow thread i would grab a frame as fast as the tracking could loop, which was about 60ms. Because at 30fps that is a new frame every 33ms, that means at maximum the image is 2 frames old, which is perfectly fine.

The reason we are on a beaglebone instead of the dashboard is i have heard stories of FTA's requiring dashboards to get shut off, and we don't want to be stuck if that happens. So we want it to be onboard.
__________________
All statements made are my own and not the feelings of any of my affiliated teams.
Teams 1510 and 2898 - Student 2010-2012
Team 4488 - Mentor 2013-2016
Co-developer of RobotDotNet, a .NET port of the WPILib.
  #10   Spotlight this post!  
Unread 19-12-2013, 23:25
Greg McKaskle Greg McKaskle is offline
Registered User
FRC #2468 (Team NI & Appreciate)
 
Join Date: Apr 2008
Rookie Year: 2008
Location: Austin, TX
Posts: 4,756
Greg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond reputeGreg McKaskle has a reputation beyond repute
Re: Fixing opencv lag

Totally understand about the approach. I have a bone at my desk for precisely this sort of experimentation. I was relating it to how LV implemented it in order to validate your approach.

I'd be interested to hear if FTAs need to limit dashboards this year. Some venue's are less wifi friendly, so it may still happen, but it is not the expected route.

Greg McKaskle
  #11   Spotlight this post!  
Unread 20-12-2013, 13:48
virtuald's Avatar
virtuald virtuald is offline
RobotPy Guy
AKA: Dustin Spicuzza
FRC #1418 (), FRC #1973, FRC #4796, FRC #6367 ()
Team Role: Mentor
 
Join Date: Dec 2008
Rookie Year: 2003
Location: Boston, MA
Posts: 1,102
virtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant future
Re: Fixing opencv lag

We did our image processing in python using OpenCV in a single thread, and didn't have any lag problems. Here are some thoughts that may help:
  • Memory allocation of large images can get expensive. Don't allocate new images each time you read something in. Instead, find the size of the image, allocate a numpy array to store the image, and process it. Reuse the same array each time you grab a new image. Similarly, when doing processing steps that return a new image (like splitting, or colorspace transforms, or whatever), allocate your buffers once, and reuse the same buffers over and over again by using the 'dst' parameter.
  • Be wary of accessing too many non-local variables in python. In our robot code (which is running python), we found that accessing a lot of things at module level instead of local level caused noticeable slowness. Presumably this is because each time python needs to do multiple lookups in each scope to find things.
  • Keep in mind that in python 2.7, because of the GIL multithreading doesn't buy you a lot unless you are I/O bound
__________________
Maintainer of RobotPy - Python for FRC
Creator of pyfrc (Robot Simulator + utilities for Python) and pynetworktables/pynetworktables2js (NetworkTables for Python & Javascript)

2017 Season: Teams #1973, #4796, #6369
Team #1418 (remote mentor): Newton Quarterfinalists, 2016 Chesapeake District Champion, 2x Innovation in Control award, 2x district event winner
Team #1418: 2015 DC Regional Innovation In Control Award, #2 seed; 2014 VA Industrial Design Award; 2014 Finalists in DC & VA
Team #2423: 2012 & 2013 Boston Regional Innovation in Control Award


Resources: FIRSTWiki (relaunched!) | My Software Stuff
  #12   Spotlight this post!  
Unread 20-12-2013, 13:52
virtuald's Avatar
virtuald virtuald is offline
RobotPy Guy
AKA: Dustin Spicuzza
FRC #1418 (), FRC #1973, FRC #4796, FRC #6367 ()
Team Role: Mentor
 
Join Date: Dec 2008
Rookie Year: 2003
Location: Boston, MA
Posts: 1,102
virtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant futurevirtuald has a brilliant future
Re: Fixing opencv lag

Quote:
Originally Posted by sst.thad View Post
I tried just requesting a single frame at a time, but it would take 100ms from requesting the image to actually receive the image. It was a little too much for me.
Were you reusing the same image buffer each time, or were you just using the returned image buffer from read()? If you are allocating a new image each time (the default behavior), that explains why you had so much lag receiving the image.
__________________
Maintainer of RobotPy - Python for FRC
Creator of pyfrc (Robot Simulator + utilities for Python) and pynetworktables/pynetworktables2js (NetworkTables for Python & Javascript)

2017 Season: Teams #1973, #4796, #6369
Team #1418 (remote mentor): Newton Quarterfinalists, 2016 Chesapeake District Champion, 2x Innovation in Control award, 2x district event winner
Team #1418: 2015 DC Regional Innovation In Control Award, #2 seed; 2014 VA Industrial Design Award; 2014 Finalists in DC & VA
Team #2423: 2012 & 2013 Boston Regional Innovation in Control Award


Resources: FIRSTWiki (relaunched!) | My Software Stuff
  #13   Spotlight this post!  
Unread 20-12-2013, 16:38
Thad House Thad House is offline
Volunteer, WPILib Contributor
no team (Waiting for 2021)
Team Role: Mentor
 
Join Date: Feb 2011
Rookie Year: 2010
Location: Thousand Oaks, California
Posts: 1,107
Thad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond repute
Re: Fixing opencv lag

Quote:
Originally Posted by virtuald View Post
We did our image processing in python using OpenCV in a single thread, and didn't have any lag problems. Here are some thoughts that may help:
  • Memory allocation of large images can get expensive. Don't allocate new images each time you read something in. Instead, find the size of the image, allocate a numpy array to store the image, and process it. Reuse the same array each time you grab a new image. Similarly, when doing processing steps that return a new image (like splitting, or colorspace transforms, or whatever), allocate your buffers once, and reuse the same buffers over and over again by using the 'dst' parameter.
  • Be wary of accessing too many non-local variables in python. In our robot code (which is running python), we found that accessing a lot of things at module level instead of local level caused noticeable slowness. Presumably this is because each time python needs to do multiple lookups in each scope to find things.
  • Keep in mind that in python 2.7, because of the GIL multithreading doesn't buy you a lot unless you are I/O bound
It works in a single thread on a computer because a powerful computer can do all the tracking faster then 33ms. When I put the code on the beaglebone, it takes about 60 ms just to run the threshold and morphology commands. So that causes the camera buffer, which gets a new frame every 33ms, to lag. if the beaglebone had more power a single thread would work, but because it does not the 2 threads are needed.

As for grabbing the image directly, openCV has no way of just opening a jpg over the network, so some other library had to be used. I was using urllib because that was the only one i could find working. And it would take 70 ms just to connect to the camera and download the image to an already allocated buffer.
__________________
All statements made are my own and not the feelings of any of my affiliated teams.
Teams 1510 and 2898 - Student 2010-2012
Team 4488 - Mentor 2013-2016
Co-developer of RobotDotNet, a .NET port of the WPILib.
  #14   Spotlight this post!  
Unread 20-12-2013, 17:04
billbo911's Avatar
billbo911 billbo911 is offline
I prefer you give a perfect effort.
AKA: That's "Mr. Bill"
FRC #2073 (EagleForce)
Team Role: Mentor
 
Join Date: Mar 2005
Rookie Year: 2005
Location: Elk Grove, Ca.
Posts: 2,384
billbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond reputebillbo911 has a reputation beyond repute
Re: Fixing opencv lag

Quote:
Originally Posted by virtuald View Post
Were you reusing the same image buffer each time, or were you just using the returned image buffer from read()? If you are allocating a new image each time (the default behavior), that explains why you had so much lag receiving the image.
I would definitely like to hear more about this approach.
I am no expert on the efficient use of OpenCV, and what you are proposing just might be the key to get rid of the minor delay we are seeing in our images. We can process 15-20 fps, but there is a small, but noticeable lag in our images.

We are using a USB camera and a PCDuino, but otherwise it is mostly the same.
__________________
CalGames 2009 Autonomous Champion Award winner
Sacramento 2010 Creativity in Design winner, Sacramento 2010 Quarter finalist
2011 Sacramento Finalist, 2011 Madtown Engineering Inspiration Award.
2012 Sacramento Semi-Finals, 2012 Sacramento Innovation in Control Award, 2012 SVR Judges Award.
2012 CalGames Autonomous Challenge Award winner ($$$).
2014 2X Rockwell Automation: Innovation in Control Award (CVR and SAC). Curie Division Gracious Professionalism Award.
2014 Capital City Classic Winner AND Runner Up. Madtown Throwdown: Runner up.
2015 Innovation in Control Award, Sacramento.
2016 Chezy Champs Finalist, 2016 MTTD Finalist
  #15   Spotlight this post!  
Unread 20-12-2013, 18:41
Thad House Thad House is offline
Volunteer, WPILib Contributor
no team (Waiting for 2021)
Team Role: Mentor
 
Join Date: Feb 2011
Rookie Year: 2010
Location: Thousand Oaks, California
Posts: 1,107
Thad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond reputeThad House has a reputation beyond repute
Re: Fixing opencv lag

Quote:
Originally Posted by billbo911 View Post
I would definitely like to hear more about this approach.
I am no expert on the efficient use of OpenCV, and what you are proposing just might be the key to get rid of the minor delay we are seeing in our images. We can process 15-20 fps, but there is a small, but noticeable lag in our images.

We are using a USB camera and a PCDuino, but otherwise it is mostly the same.
Does the lag show up before processing or after processing. If it shows up before processing that could be a solution. If it shows up after processing then its the actual processing time that could be causing the lag. One way I liked to test this is to put a timestamp at the beginning of the processing, then another at key points in the processing. Then you can calculate the difference and see how much time each process is taking. I used this on ours to see that the morphology and thresholding took about 50ms to do on a 320x240 image. This would cause any processed image to be more behind then that, at which point is easily visible.
__________________
All statements made are my own and not the feelings of any of my affiliated teams.
Teams 1510 and 2898 - Student 2010-2012
Team 4488 - Mentor 2013-2016
Co-developer of RobotDotNet, a .NET port of the WPILib.
Closed Thread


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 03:21.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi