New guy needs some direction please!...

[bradleyheathhay]

Hey, can't say how grateful I am to have found this forum! Although I'm not from the robotics community, I am a crafter of sorts and have an idea for a little device I'd like to build, and I need your advice on where to go from here.

My task is fairly simple. I work in an industry that uses 6-8" computer tablets, and when a part of the screen changes color to indicate an available 'job,' whoever hits the on-screen 'button' first gets the job, thusly increasing their income. Basically, I just need to create a device that senses when that particular part of the screen changes color, and then automatically responds with some type of actuator (I assume) to make contact with the screen using a stylus device.

Sounds simple? I hope so!

I know this may take some work, but I'm def willing to make an investment in time and effort.

Without going into further detail I should stop short and simply ask...is this the place I need to be to begin developing such an idea? Also, I'd like to try to develop this idea simultaneously at as many locations as possible here on the net; so if there's another place besides here that may be of help I would surely appreciate some direction.

Looking forward to your ideas and questions!

[ctt956]

This could probably be done fairly easily with an Arduino. You'd just need a color sensor and the stylus could be on a servo. Program it so when it sees a color, the servo moves to tap the screen, then back to its original position. I haven't worked with color sensors, but servos are easy to use with Arduinos. This is a good place to get help, and I'm sure others will chime in! Most of the people here are nice and helpful. Here's something I made using a RedBoard Arduio and the servo linked above.

[Hitchhiker 42]

I'm not sure exactly how, but I feel like directly accessing the screen's color would be more accurate and less error-prone than a color sensor, which could be affected by lighting conditions, etc. I'd love to see how this could be done (or if it can't, how it can't).

[Rangel(kf7fdb)]

Originally Posted by Hitchhiker 42

I'm not sure exactly how, but I feel like directly accessing the screen's color would be more accurate and less error-prone than a color sensor, which could be affected by lighting conditions, etc. I'd love to see how this could be done (or if it can't, how it can't).

Lighting conditions is probably the biggest issue. To the op, one way to reduce this would be to make an enclosure around the sensor so that when you place it on the tablet screen, it's the same lighting condition every time. Essentially the only light going into the sensor would be the light from the screen. This should work unless the color that the screen changes to is very similar to previous color. Also agree with using an arduino and a servo. I don't have any specific recommendations on color sensors though.

[adciv]

Alright, lets say "Red" is no job and "Green" is job. I'm also assuming the buttons for this are in fixed locations on the screen.

Setup all the tablets on a table, point a USB webcam at the tablets. Apply an HSV filter to what the camera sees to detect Green. All buttons are maped to a grid that corresponds to the pixels in the camera. In this way, when green is detected, the system knows which button is green.

Hard part: Get a device that lets you replicate a human touch. Stick these over every button on the screen. Whenever you get "Green" at that button, activate the device corresponding to it.

[Hitchhiker 42]

Originally Posted by adciv

Alright, lets say "Red" is no job and "Green" is job. I'm also assuming the buttons for this are in fixed locations on the screen.

Setup all the tablets on a table, point a USB webcam at the tablets. Apply an HSV filter to what the camera sees to detect Green. All buttons are maped to a grid that corresponds to the pixels in the camera. In this way, when green is detected, the system knows which button is green.

Hard part: Get a device that lets you replicate a human touch. Stick these over every button on the screen. Whenever you get "Green" at that button, activate the device corresponding to it.

For the device that replicates human touch, you might just want one of those gloves that let you use your phone like this: https://www.amazon.com/Hi-Fun-HFHiCA.../dp/B009NBRXLQ. Use one finger for this.

[DaveL]

If the target is in a fixed location and no one is counting false touches,
you could simplify the design by constantly tapping the screen, like the example given above.

[Monochron]

Originally Posted by adciv

Hard part: Get a device that lets you replicate a human touch. Stick these over every button on the screen. Whenever you get "Green" at that button, activate the device corresponding to it.

Not that hard

[ctt956]

Originally Posted by Monochron

Not that hard

That makes sense, considering that sausages have salt which conducts electricity...

[astrayelmgod]

Well, yes, but I think you're all missing the point. When the spot changes color, any worker WHO IS READY can press the button. You're all slaving a mechanical finger to the spot; regardless of whether the operator is ready or not.

One part that got left out is, how often are these jobs presented? I've never worked on a piece work basis, but my impression is that they would be presented very fast. Fast enough that no operator could be ready every time for all them.

Another question is what else are these tablets used for? Since the operator apparently is not waiting for physical stuff to be delivered to the work station (as in sewing clothing together), is the work also done on the tablet? If so, that limits the hardware you can have hovering over the screen.

And of course, when all the operators have these goodies, then you are right back where you are now, taking pot luck.

Also, there are better web sites than this one for this kind of project. "element14.com"comes to mind. Search for "maker movement" for more contacts.

[Aeglos]

A suggestion that I would have to actuate the button, at least if you are using a servo, would be to use a cam linkage to depress the button on the screen. If you set it to move to an angle past the actuation point when a color change is detected, it will "tap" the screen.

It's like the side of a train, sort of, in how it works- I hope the picture I've attached helps.



The output shaft is at the top of the image, and the spring holds the output shaft against the cam, so that it follows any grooves/ depressions/ rises in the cam surface; for example, rotating this cam counter- clockwise would cause the shaft to actuate, and then pull back, beginning to actuate again as it nears the elongated end of the cam when it has gone a full revolution.

[dirtbikerxz]

Originally Posted by astrayelmgod

Well, yes, but I think you're all missing the point. When the spot changes color, any worker WHO IS READY can press the button. You're all slaving a mechanical finger to the spot; regardless of whether the operator is ready or not.

Well the OP did say that he wanted a device to hit the button as fast as possible. He didn't say anything about not being ready...

And if he wasn't ready, well maybe he can build in a switch that disables the mechanism.