Chief Delphi

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)
-   General Forum (http://www.chiefdelphi.com/forums/forumdisplay.php?f=16)
-   -   Spambot Prevention Suggestions (http://www.chiefdelphi.com/forums/showthread.php?t=118016)

jwallace15 26-07-2013 20:47

Spambot Prevention Suggestions
 
Joe Ross suggested the creation of this thread to discuss ways to stop the recent flooding of spam onto Chief Delphi.

http://www.chiefdelphi.com/forums/sh...d.php?t=115858

Would this have to do with anything?

brennonbrimhall 26-07-2013 20:53

Re: Spambot Prevention Suggestions
 
Here's a re-post of what I posted in the soon-to be deleted thread:

Quote:

Originally Posted by DampRobot (Post 1284714)
Here's an idea, verify that all new accounts are humans by adding an image verification thingy?

I would be surprised if they don't already, as they require one for every search if you are not logged in.

Quote:

Originally Posted by Koko Ed (Post 1284715)
A better one would be track these jerks down by their IP address and zapping their computers into oblivion so they can't terrorize anyone else with their useless nonsense!

I agree. Minus the zapping and oblivion business. ;)

A thought experiment: the problem becomes that the spammer will simply look at the image verification for the bot account they set up; e.g., do it manually. Also, to evade IP address detection, wouldn't they just go to some other public network?

Security-wise, every measure you take is breakable – take hashing, for example. Even though it's designed to be a one-way street with next-to-zero odds of collision, breaking them is still possible. However, it is designed to only brake under an insane amount of computational effort and expenditure of resources.

The problem with the current methods used to prevent bots is that they are all easily defeated with a small cost in resources – it takes a minute for the spammer to write down the image verification. IP address blacklisting is perhaps an order of magnitude harder to break – it probably takes, on average, 40 minutes to get to a local library plus the cost in time to generate an account. You could improve this by adding a cookie to the browser that generated the post the next time they come to CD (with a nice, graciously professional ban message, of course) that tells vBulletin to exclude the new account they are creating. But this would end as soon as the spammer cleared their cookies.

That's the problem with spamming – the more security you put in for prevention, the harder it is for your actual users to get stuff done.

One feature that I would suggest is having a team contact that has to approve all accounts that are attempting to register for the team, in a manner like TIMS/STIMS. While it certainly wouldn't apply to bots without a team, it would help the Juggernauts' number from being abused all the time.

------------------------------

And my response to Joe Ross' link to spam detection software:

Quote:

Originally Posted by Joe Ross (Post 1284732)
There is: http://www.chiefdelphi.com/forums/sh...hlight=akismet



There is both an image verification as well as an additional textual verification question.

Has this been relaxed lately? This particular bot in question was created this month. And has only posted once.

nicholsjj 26-07-2013 21:03

Re: Spambot Prevention Suggestions
 
One thing I think would help would be to add moderator approval for a first post by a new user that doesn't claim a team

efoote868 26-07-2013 21:12

Re: Spambot Prevention Suggestions
 
Require new accounts to receive at least some positive reputation before they're allowed to create a new thread, which is where most spam goes.

If all of us veterans know about this policy, I'm sure we'd be more than happy to keep an eye out for new accounts and rep them for making any contributing post.

If it isn't too difficult, you could also have a 30 day waiting period for the account to do something relevant to a legitimate user, such as using the search function or browsing through several threads.

The point is that these simple methods could deter automated spammers, but wouldn't be too much of a hassle for a human user. And if it is a human that is setting these accounts up, there isn't much you can do to prevent it; although you could get more moderators to police the forums for spam.

EricH 26-07-2013 21:14

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by nicholsjj (Post 1284740)
One thing I think would help would be to add moderator approval for a first post by a new user that doesn't claim a team

Also trivial to get around. All users claim a team in some way--if you'll notice, I'm currently set to team 0000 (unaffiliated/other). It's not hard to put in something about team 0001 or 1234 or some other team, real or not.

Now, there is/was some sort of that thing set up a while back, IIRC, but it was automated or semi-automated. Might be time for Brandon to take a look at some of those parameters and see if they can be adjusted.

z_beeblebrox 26-07-2013 21:15

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by nicholsjj (Post 1284740)
One thing I think would help would be to add moderator approval for a first post by a new user that doesn't claim a team

Or for any new user?

Perhaps eliminate the delay between allowing posts and emails for people reporting spam... I'm not sure how that could practically be done. Maybe based on # of posts or reputation?

Filter posts from new users with lots of hyperlinks. You rarely see spam without a big block of links at the bottom.

jwallace15 26-07-2013 21:15

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by nicholsjj (Post 1284740)
One thing I think would help would be to add moderator approval for a first post by a new user that doesn't claim a team

The problem is that some of the bots claim teams. A couple that I have seen calim to be from team 3 (both of which set their location to "india"), I think one from team 1, etc. EDIT: EricH beat me to saying this

Quote:

Originally Posted by z_beeblebrox (Post 1284744)
I'm not sure how that could practically be done. Maybe based on # of posts or reputation?

Some spambots post on random threads responses that don't make any sense to gain posts. That way they seem less likely to be a spambot. For instance, I've seen a couple posts from spambots saying "That is good advice" or "This will be beneficial to my well-being" or things like that. Recently one copied word-for-word what someone posted at the beginning of a thread and reposted it.

I do think that it is a good idea for a moderator to approve a new user's first post, but it is kinda hard to weed out the good from the bad. Not to mention it would take up a lot of the CD Moderator's time.

EDIT: I'm sure Koko Ed would love to see someone create a notspambot that goes to where the spambots hang out and post relevant things. That will teach them.

coalhot 26-07-2013 21:18

Re: Spambot Prevention Suggestions
 
My original post:


Quote:

Originally Posted by coalhot (Post 1284727)
It shouldn't be terribly hard to make an autodetect script. All the bot spam posts in the past few months have been in the same manner. A line of text, and then a whole stack of links. All posted by a relatively new account, not associated with a team, and not properly set up...


Also, a thought. What if the forum was set up to pull a thread if enough people reported it, and it was posted by someone "questionable" (new user, no rep, low post count), and the thread would only be reinstated after a moderator approved it?

jwallace15 26-07-2013 21:24

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by coalhot (Post 1284748)
Also, a thought. What if the forum was set up to pull a thread if enough people reported it, and it was posted by someone "questionable" (new user, no rep, low post count), and the thread would only be reinstated after a moderator approved it?

Such as this? I reported this, nothing happened, then the person added to the spam and someone else reported it. And it's still there.

DampRobot 26-07-2013 21:28

Re: Spambot Prevention Suggestions
 
OK, if image verification or IP blacklisting won't work, why not ask a FIRST related question. For example, what was the 2011 game name? Or, Which country are 1114 and 2056 located in? Or, what is one FRC supported programming language?

If the spammers know enough about robotics to answer those questions, then I can't see why they would want to spam CD.

The other suggestion is to make Mods look at a posters first post before allowing them to post.

EricH 26-07-2013 21:31

Re: Spambot Prevention Suggestions
 
I'm wondering, as I have on occasion, how many of the listed moderators are actually active any more. Personally, a quick scan down the list of moderators shows that, for about half of them, I haven't seen a post from them in quite some time, or they only moderate one or two subforums.

I realize that I don't see all the moderator activity by any means, and actively posting isn't necessarily the best means of finding an active moderator... but I suspect it may be time for the CD admin team to review the moderator list and assign some of them a couple extra subforums or something like that.

Reports don't do any good if they're sitting in a PM box that isn't monitored by at least a semi-active moderator. Wonder if that's at least part of the problem...

Gregor 26-07-2013 21:32

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by DampRobot (Post 1284752)
OK, if image verification or IP blacklisting won't work, why not ask a FIRST related question. For example, what was the 2011 game name? Or, Which country are 1114 and 2056 located in? Or, what is one FRC supported programming language?

As hard as it to believe, a lot of FIRST participants wont know the answer to those questions, especially people making new accounts.

Woolly 26-07-2013 21:34

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by Gregor (Post 1284754)
As hard as it to believe, a lot of FIRST participants wont know the answer to those questions, especially people making new accounts.

Fill in the blank with the missing word in the FIRST acronym?


I mean, even if they are new members, if they don't know stuff like that, it maybe it's time they do a Google search and learn it.

EricH 26-07-2013 21:39

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by Woolly (Post 1284755)
Fill in the blank with the missing word in the FIRST acronym?


I mean, even if they are new members, if they don't know stuff like that, it maybe it's time they do a Google search and learn it.

If a new member can do a search and learn it, so can a spammer. As I recall, there IS such a question (having to do with a core value of FIRST) already.


Captcha? Check.
FIRST-related question? Check.
Auto-moderator/quarantine? Check.

Anybody got any other ideas? BTW, these were all implemented either early on or after a particularly vicious spam attack.

DampRobot 26-07-2013 21:49

Re: Spambot Prevention Suggestions
 
Not allowing first time posters to start a thread. If you are a real FIRSTer, you must have something to say in a thread before you start your "PLEZ HELP ROBOT SMOKING" thread. It would encourage searching too!

jwallace15 26-07-2013 21:51

Re: Spambot Prevention Suggestions
 
Although strange locations and non-FIRST postings are among the activities of spambots, not all people who join CD that do those things are spammers.

http://www.chiefdelphi.com/forums/sh...d.php?t=117653

Couple weeks ago, this person not affiliated with FIRST asked a viable programming question.

From what I've seen spambots only spam when they start their own thread, not on an existing thread. My suggestion is to pre-screen new thread requests from new users.

I also like Efoote868's suggestion about basically granting more priveliges to those with higher rep.

I recall starting a thread when I first joined and seeing a message that said my thread would be previewed by Moderators before it was posted, and that I kept checking to see if it was approved. Was this feature taken away?

EDIT: I recall once seeing a viable thread started by a new user that was edited once it was posted to be spam. So the system can be fooled.

connor.worley 26-07-2013 21:57

Re: Spambot Prevention Suggestions
 
A lot of the spam threads contain 15+ links. Maybe a hyperlink limit/verification would be ideal?

Caleb Sykes 26-07-2013 22:01

Re: Spambot Prevention Suggestions
 
Quote:

Require new accounts to receive at least some positive reputation before they're allowed to create a new thread, which is where most spam goes.
I disagree. I made my CD account specifically to ask a time-sensitive question during the build season. There is no way that I would have even bothered if I had had to weigh in on other discussions and wait for someone nice to give me rep points. I had, in fact, searched through CD several times, and my question hadn't been asked.

Quote:

OK, if image verification or IP blacklisting won't work, why not ask a FIRST related question. For example, what was the 2011 game name? Or, Which country are 1114 and 2056 located in? Or, what is one FRC supported programming language?
I didn't learn about 1114 and 2056 until the end of the 3rd year on my team. "Common knowledge" on CD is, quite frankly, not common knowledge for a lot of students/mentors out there.

techhelpbb 26-07-2013 22:21

Re: Spambot Prevention Suggestions
 
Ask FIRST for the list of mentors they use to control access to the official question forum.
Then ask those mentors to register at ChiefDelphi.
Then give them the ability to confirm team member registrations.

No confirmation ... no post without moderation.

Set up modrewrite on this domain.
Track inbound IP.
Anyone claiming to be one place posting from more than 1 adjacent state/province not mentor approved gets flagged.
Saves moderation time.

Use Bayesian rules to flag posts.

Any post flagged by 2 confirmed mentors is redacted prior to review.

Hyperlinks are shortened. Checked by SourceFire, RBL and local black list.
This allows caching as well.
Any non-confirmed poster with less then 10 posts no hyperlinks allowed.
Allow any confirmed users to flag a hyperlink as a threat then setup a warning or make it text.

Allow donations over $15.
Allow donations of FIRST items ChiefDelphi can resell.
Allow donors to select to be listed if they like.

Expire confirmed accounts after 4 years.

efoote868 26-07-2013 22:27

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by inkling16 (Post 1284762)
I disagree. I made my CD account specifically to ask a time-sensitive question during the build season. There is no way that I would have even bothered if I had had to weigh in on other discussions and wait for someone nice to give me rep points. I had, in fact, searched through CD several times, and my question hadn't been asked.

Asking a new question might not require a new thread; especially if there were one stickied for that purpose.

magnets 26-07-2013 22:35

Re: Spambot Prevention Suggestions
 
My first couple posts took about a day to go through because it said that a moderator was approving it. From what I see, this only happens to a few of the new users, or moderators are approving spam. Perhaps this policy should be tightened to the first ten posts from any users.

Scott L. 26-07-2013 22:37

Re: Spambot Prevention Suggestions
 
The use of a verisign generated token that changes everytime the use logs in. there is a phone apps from verisn to work as the token. this would lower the amount of repeat spammers, since the code changes each time based on an algarith, and changes every 30sec. :D

Chris is me 26-07-2013 22:47

Re: Spambot Prevention Suggestions
 
The question doesn't have to be hard to thwart spambots. They are not targeting Chief Delphi in particular - the bots sweep the web and post to any vBulletin forum they can find. If the registration question is as simple as "Who founded FIRST?" or "What is the acronym for FIRST Robotics Competition?" it would probably work.

Honestly, there is no need to be needlessly restrictive and make this community harder to enter just because there's a handful of spam threads a day.

jwallace15 26-07-2013 23:08

Re: Spambot Prevention Suggestions
 
[Disclaimer]

For the record I don't expect this idea to be taken seriously, but as a joke.

[/Disclaimer]

Let's create a spam forum for the bots to post on. And we just won't read it! That way they get to advertise but it doesn't show up on the stream on the main CD page.

Problem solved!

EricH 27-07-2013 02:15

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by techhelpbb (Post 1284764)
Ask FIRST for the list of mentors they use to control access to the official question forum.
Then ask those mentors to register at ChiefDelphi.
Then give them the ability to confirm team member registrations.

Not happening, for the following reasons:
1) Probably half the mentors won't register, unless it's required (and FIRST won't require it, I'm sure). And confirming team member registrations? Har.
2) The odds of a spambot using a given team number are slim--but the odds of them using 0000 are pretty decent.
3) Restricts Chief Delphi to only FIRSTers--which doesn't help with growing FIRST at all.

Essentially, that places a burden on the mods the first few times any given new member posts. As I noted earlier, I think the mod list needs a shakeup before any more work is placed on them.

The location tracking, and mentor approval... I think that's a bit clunky.

Oh, wait, I know! What's the most current version of vBulletin again, and which version are we on? (hint: they aren't the same)

techhelpbb 27-07-2013 08:37

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by EricH (Post 1284785)
Not happening, for the following reasons:
1) Probably half the mentors won't register, unless it's required (and FIRST won't require it, I'm sure). And confirming team member registrations? Har.
2) The odds of a spambot using a given team number are slim--but the odds of them using 0000 are pretty decent.
3) Restricts Chief Delphi to only FIRSTers--which doesn't help with growing FIRST at all.

Essentially, that places a burden on the mods the first few times any given new member posts. As I noted earlier, I think the mod list needs a shakeup before any more work is placed on them.

The location tracking, and mentor approval... I think that's a bit clunky.

Oh, wait, I know! What's the most current version of vBulletin again, and which version are we on? (hint: they aren't the same)

I think you misunderstood. The confirmed users get full access with no restriction immediately. Non-confirmed members get some restrictions.
So if your team leader will not bother you still can post. Just deal with some restrictions initially. Ten post limits on hyperlinks exist on other forums. Membership durations on certain features exist on other forums. If FIRST will not provide the list or as you said the team leaders are not cooperating then allow cross team confirmation.

IP location tracking not only works I personally implemented it on sites that control 10% of the world's finances. You just need to be clever with it. You never use it as a completely automated blacklist. You use it to get a feel for who your customers are and their consistency as a moderation tool. If you know it is unusual for users to pop in from China then flag those posts. Moderation is *so* much less work when you mine your data intelligently. If the moderation is a problem now. What would happen if the post count doubles? In theory it is desirable to grow FIRST so a doubled post count should be positive but if as a moderator the job is too big now....well?

artdutra04 27-07-2013 09:55

Re: Spambot Prevention Suggestions
 
99.9% of the moderating I do is deleting spam. So here are some points:

1. For a while before last build season, all posts from new users had to approved by a moderator. On average, there seemed to be a 24-48 hour backlog on posts. Some users would then try to post again... and again... and again... and again because they did not see the posts show up immediately. This created almost as many problems as it solved.

2. The most viable method to preventing spam is to prohibit users that have under 50 posts OR less than 300 points (three green bars) in reputation from:
- Posting hyperlinks in posts or signature
- Editing their own posts

3. Some sub-forums have like 50 moderators, so chances are higher that someone will delete it sooner. Other really obscure and hardly ever used sub-forums have like 3 total moderators, and spam posts there can linger for months and months. I suggest we both significantly prune/lock/archive these forums and add some more active moderators to the remaining forums.

I've seen some spammers go on a 10-20 post blitz of posting worthless drivel posts to get their post count up. 50 posts and 300 points should be a high enough threshold to discourage most spammers. And no hyperlinks of any kind make spam posts worthless.

Maybe to help get some extra money, users under 50 posts or 300 rep points that donate to CD can have these restrictions immediately removed.

artdutra04 27-07-2013 09:58

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by Chris is me (Post 1284770)
The question doesn't have to be hard to thwart spambots. They are not targeting Chief Delphi in particular - the bots sweep the web and post to any vBulletin forum they can find. If the registration question is as simple as "Who founded FIRST?" or "What is the acronym for FIRST Robotics Competition?" it would probably work.

Honestly, there is no need to be needlessly restrictive and make this community harder to enter just because there's a handful of spam threads a day.

Most of the new spambots actually rely on a human somewhere (usually somewhere halfway across the world) to confirm any tricky things in the registration process, then use bots to post.

E Dawg 27-07-2013 12:40

Re: Spambot Prevention Suggestions
 
Make it so that you must have made at least one post on CD before you can start a thread. That way no one is blocked if they need help, but spambots cannot get in as easily.

techhelpbb 27-07-2013 12:41

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by artdutra04 (Post 1284801)
99.9% of the moderating I do is deleting spam. So here are some points:

1. For a while before last build season, all posts from new users had to approved by a moderator. On average, there seemed to be a 24-48 hour backlog on posts. Some users would then try to post again... and again... and again... and again because they did not see the posts show up immediately. This created almost as many problems as it solved.

That works much better when the yet to be released posts are merely redacted.
Just post a place holder: Awaiting post approval. At least send an e-mail to the poster.

Then people know what is going on and not flood you.

You could do away with hyperlinks if you snapshot the target website.
Think like Google cache which is a spider.
You can get a picture of the canvas object in Mozilla and Chrome browsers.
Though this could get costly.

protoserge 27-07-2013 14:55

Re: Spambot Prevention Suggestions
 
Is there any possibility of updating the forum software to the current version 5?

Here is a list of plugins to help prevent spambots: http://www.vbulletin.com/forum/forum...ation-requests

Cory 27-07-2013 15:00

Re: Spambot Prevention Suggestions
 
I think if users under a post count threshold post more than maybe 3 (insert amenable number here) links in their post/signature they should automatically get sent to moderation.

The timely removal of spam is definitely because there's very few moderators in some of the forums that weren't getting spammed until recently. I know more moderators were added a year or two ago, but it didn't really help much (I think only one or two of them is actually still active on Chief). I take down stuff whenever I get the notification but anything in obscure forums takes longer since there's only a handful of active moderators as Art said.

Akash Rastogi 27-07-2013 15:11

Re: Spambot Prevention Suggestions
 
How does one volunteer to help moderate the forums?

Pault 27-07-2013 15:13

Re: Spambot Prevention Suggestions
 
I think if we add on some moderation to posting links it should include a list of approved websites that don't need to be checked. Most links on here lead to either chiefdelphi.com or youtube.com, so by saying those are automatically approved would really lessen the burden on moderators.

Chris is me 27-07-2013 15:30

Re: Spambot Prevention Suggestions
 
In VB there is also the possibility of making users Super Moderators. This allows them to delete spam in any forum but stops short of giving the user Administrator privileges. I don't know how many of those CD has, but that might be helpful for spam removal purposes.

I also like Pault's suggestion: Make a whitelist of allowed links for posters who have under X posts and / or have been around for less than Y days. If the users need to link to something else, I'm sure we could just copy and paste a plaintext URL to see what the user is talking about. Not having hyperlinks for new users is a small price to pay that would really help I think.

It's cool to see this community come together to brainstorm solutions to problems like these. I'm happy to help however possible with the spam problem.

jwallace15 27-07-2013 15:37

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by Chris is me (Post 1284830)
In VB there is also the possibility of making users Super Moderators. This allows them to delete spam in any forum but stops short of giving the user Administrator privileges. I don't know how many of those CD has, but that might be helpful for spam removal purposes.

It would be a lot more useful than reporting everything for a couple people to delete it. Perhaps everyone who has rep level "reputation beyond repute" (I'm not sure how many points that is, I'm assuming either 1500 or 2000) or something like that could be granted this privilege.

I'm sure those with lower reps would like to volunteer to do that, too. I know I wouldn't mind helping out.

Pault 27-07-2013 16:07

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by jwallace15 (Post 1284832)
It would be a lot more useful than reporting everything for a couple people to delete it. Perhaps everyone who has rep level "reputation beyond repute" (I'm not sure how many points that is, I'm assuming either 1500 or 2000) or something like that could be granted this privilege.

I'm sure those with lower reps would like to volunteer to do that, too. I know I wouldn't mind helping out.

I like the idea, but something just doesn't feel right to me giving that much power to that many people who aren't manually approved. I'm sure it would be sometimes used against posts that someone thinks are un-gp, which isn't right. So here's my revision of that idea:

People with a full rep bar or other approved people gain the privilege to flag a post . If 3 people flag a post, it is automatically moved to a separate forum that only moderators can see. The mods can then go through that forum and delete all actual spam posts. If they come across something that isn't supposed to be there, they can return it to where it once was and issue a warning to the people who flagged it. 2 warnings and that person loses his/her privilege to flag.

One other benefit of this system is that there would be no more problems where the spam is posted in a forum without any active moderators.

DonRotolo 27-07-2013 20:07

Re: Spambot Prevention Suggestions
 
it is possible in VB to restrict users with certain characteristics* from posting links. I would venture to say that is isn't spam unless it has hyperlinks.

*For example, number of posts, rep points, and so on.

magnets 28-07-2013 12:38

Re: Spambot Prevention Suggestions
 
Here's another idea.

I'm not sure if VB supports this, but another forum I used to visit had an option to flag as spam. Then, if ten users with more than 100 posts had marked a post as spam, it would disappear, and a moderator could either delete it if it was spam, or keep it if it was a real post

With this solution, the spam would show up, but it would disappear as soon as ten users marked it as spam.

protoserge 28-07-2013 13:19

Re: Spambot Prevention Suggestions
 
On that note, any user with say 600+ rep points on this forum knows what spam on this forum is and could mark it as spam, immediately quarantining the post. This would only require one member to perform. The moderator could then delete/ban/block the offending account.

cbale2000 28-07-2013 14:06

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by artdutra04 (Post 1284801)
1. For a while before last build season, all posts from new users had to approved by a moderator. On average, there seemed to be a 24-48 hour backlog on posts. Some users would then try to post again... and again... and again... and again because they did not see the posts show up immediately. This created almost as many problems as it solved.

I'm guilty of having done this before myself on a different forum (though on that forum doing it too often could get you banned).

This happens when it's not immediately clear to new users why their posts aren't showing up. Typically forums tell you your post is awaiting approval in the redirect page after you click "submit", but honestly, no one ever reads those. What you need is a bright red font message right above the "submit" button telling you that your post will need to be approved.


That all said, I do have a few thoughts that might help reduce spam (some of these have already been mentioned but are worth repeating) ...
  • First 5-10 posts a user creates require a CAPTCHA to post (prevents human registration - bot posting)
  • Add a generic FIRST-related question to the registration (if it already had one, perhaps add a second).
  • Make threads reported by multiple (~100+ post) users get moved to an invisible moderation forum so they're not seen while awaiting mod review.
  • Block hyper-linking from users with less than 5-10 posts.

JohnFogarty 28-07-2013 19:09

Re: Spambot Prevention Suggestions
 
I recommend this...I use it on all of my websites that have forum components. So far ZERO spam.

http://confidenttechnologies.com/pro...fident-captcha

/Profit

protoserge 15-08-2013 08:32

Re: Spambot Prevention Suggestions
 
How about this one?


http://areyouahuman.com/home-2/?utm_...om%2Fhome-2%2F

Require 3 or 10 of them.

Al Skierkiewicz 15-08-2013 09:07

Re: Spambot Prevention Suggestions
 
I guess it would be appropriate to let you know that there are many moderators that watch and receive alerts. When you report a post, we receive an email alert and respond if we are on line. Also some of us watch the new users link on the CD homepage and investigate. Some spammers are caught before they ever post anything because they have certain items in their online profile that highlights them. Sometimes this is a hyper link in their signature line, sometimes it is a website in their profile and sometime it is the nature of their screen name. Some bots are now filling in the field with a random team number so if I see an FRC team number greater than 5000 it is a red flag. Also if I see a retired team number like 47 I also suspect a bot.
I check new users every few days and eliminate as I find them. Several of us start early in the morning so we clean up when we get to work. Eastern time zone usually gets there first like John and Gary. I follow at 6 CDT. Others who are on later in the day like Mark and others will get things while the rest of us are getting home and having dinner and still others will get stuff late night like BillFred.
However, we don't all have the same rights, so while we get the report, I can't delete post in all forums but I can delete users so they can't post again. Some bots (not a lot) have managed to post 20 times in a few seconds.

EricH 04-09-2013 20:16

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by cbale2000 (Post 1284954)

That all said, I do have a few thoughts that might help reduce spam (some of these have already been mentioned but are worth repeating) ...
  • Block hyper-linking from users with less than 5-10 posts.

Just to bring something up right quick, this would have to apply to all edits as well as original posts.

I saw a spammer put up a thread title that didn't make sense... with identical post content... but no links. 5 minutes later, the same post had MORE words that didn't make sense, plus the "normal" 30 or more links. (Another 5 minutes later, before this post you're reading hit the forums, the thread in question was gone, thanks to the mods.)


Just a heads up that we may have an invasion of editing spambots--make post, edit post with links.

jwallace15 04-09-2013 20:26

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by DonRotolo (Post 1284866)
I would venture to say that is isn't spam unless it has hyperlinks.

http://www.chiefdelphi.com/forums/sh...d.php?t=117091

Quote:

Originally Posted by EricH (Post 1289939)
I saw a spammer put up a thread title that didn't make sense... with identical post content... but no links. 5 minutes later, the same post had MORE words that didn't make sense, plus the "normal" 30 or more links

I've been seeing that too. There was one (can't remember when, but very recently) that had a bunch of text in {}'s.

Another thing I've noticed is that generally speaking a thread title that has something to do with a celebrity, accident, or something having to do with Middle Eastern politics is usually spam.

sanddrag 14-10-2013 02:42

Re: Spambot Prevention Suggestions
 
Every day for the last week seems like many spammer hits here. I don't want this great site to be ruined by spammers. Anything we can do to step up the fight? More mods?

I'm happy to mod if needed. I'm west coast so always up and on here late. And I think with over 11 years on the forums, I'm qualified enough. Let me know if I can help.

Cory 14-10-2013 02:55

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by sanddrag (Post 1296273)
Every day for the last week seems like many spammer hits here. I don't want this great site to be ruined by spammers. Anything we can do to step up the fight? More mods?

I'm happy to mod if needed. I'm west coast so always up and on here late. And I think with over 11 years on the forums, I'm qualified enough. Let me know if I can help.

Adding mods might be helpful, but it's reactionary. There needs to be a way to head it off before it happens.

I thought someone mentioned there was a newer version of vBulletin with better spam blocking, but maybe that was awhile ago and we're already using that.

JohnBoucher 14-10-2013 07:05

Re: Spambot Prevention Suggestions
 
A hidden field could be added to the registration page. A spambot will fill it in but a human won't see it and leave it blank. If hidden field is filled then reject.

Racer26 14-10-2013 11:10

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by JohnBoucher (Post 1296297)
A hidden field could be added to the registration page. A spambot will fill it in but a human won't see it and leave it blank. If hidden field is filled then reject.

That... might actually work.

artdutra04 14-10-2013 11:21

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by Cory (Post 1296275)
I thought someone mentioned there was a newer version of vBulletin with better spam blocking, but maybe that was awhile ago and we're already using that.

The version of vBulletin that Chief Delphi currently runs on was released in 2006.

Joe Ross 14-10-2013 12:03

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by JohnBoucher (Post 1296297)
A hidden field could be added to the registration page. A spambot will fill it in but a human won't see it and leave it blank. If hidden field is filled then reject.

Most spambots don't fill all fields, so that wouldn't work.

The spambot already needs to know to type "inspiration" or "science" into a field when prompted, or answer what Dean Kamen's first name is, or answer what day of the week it is. Presumably, it's not a spambot doing that part of registration.

artdutra04 14-10-2013 15:46

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by Joe Ross (Post 1296351)
Most spambots don't fill all fields, so that wouldn't work.

The spambot already needs to know to type "inspiration" or "science" into a field when prompted, or answer what Dean Kamen's first name is, or answer what day of the week it is. Presumably, it's not a spambot doing that part of registration.

I don't think we're actually dealing with spambots anymore.

A lot of spammers have realized its easier to stay one step ahead of anti-spam measures by paying someone in Russia/China/southeast Asia/other developing countries a few bucks a day to sit and manually enter everything.

Look at all the timestamps of the spam posts and threads; they are all usually late at night for North America and working hours in the countries listed above.

yash101 15-10-2013 01:24

Re: Spambot Prevention Suggestions
 
1 Attachment(s)
Here is a screenshot of some spam I am getting on my website. What do you notice? I noticed that the email addresses are random letters and numbers and the names are the same category. This could be a very crude and a first way to filter out Spam. However, I believe that this forum uses Askimet. As soon as I installed and activated that, I didn't have to worry about spam, and in face, I just disabled the requirement for comment moderation because I haven't had a single spam get through.

In Short, If an email is just a weird string of character, have the user trying to sign up have to go through more recognition, same with the name of the user.

Also, I can imagine that a spammer might, himself creating an account/set of accounts for the spambots to use to get into the system. Then, the user can be disabled by a human who can detect that something is spam.

Also, Why do you have those two ads at the top. They never change, so they become kind of annoying, and I am pretty sure the "donate" button has been clicked many times because of how useful this forum is :D

EricH 15-10-2013 19:29

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by yash101 (Post 1296511)
Also, Why do you have those two ads at the top. They never change, so they become kind of annoying, and I am pretty sure the "donate" button has been clicked many times because of how useful this forum is :D

You'd be surprised. I think the "donate" button was up before the ads were.

Note: One of the two ads is ALWAYS from a forum sponsor (IFI), and it's rarely the same ad two pages in a row. The other is typically from a sponsor of team 51 (used to be 47) who wants to get their message out. Again, rarely the same two pages in a row.

Easy way to turn them off: become a forum subscriber for a couple years. (Translation: big-ish donation.)

Navid Shafa 16-10-2013 17:27

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by sanddrag (Post 1296273)
I'm happy to mod if needed. I'm west coast so always up and on here late. And I think with over 11 years on the forums, I'm qualified enough. Let me know if I can help.

I'll second that. I check CD at least once a day and I'm also on at obscure times. Happy to help.

jwallace15 16-10-2013 17:38

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by JohnBoucher (Post 1296297)
If hidden field is filled then reject.

This could work.

(a little off topic) Our team leader put something like that on the bottom of our last team email; it was a Google Form that you filled out with your name and other emails you wanted the team emails to go to. It was also a test to check who was checking and fully reading the team emails and who wasn't (it also checked who viewed it within 24 hours).

Whippet 16-10-2013 17:56

Re: Spambot Prevention Suggestions
 
Perhaps anyone with an IP address from a country where no FIRST teams exist should have to be manually approved by a mod before they could post?

Nuttyman54 16-10-2013 18:06

Re: Spambot Prevention Suggestions
 
One thing I've noticed is that most of the spam has upwards of a dozen hyperlinks in it (most somewhere around 20). They also all seem to appear as new threads.

Would it be possible to put an additional level of security/captcha/mod approval type thing on any new threads that get started with more than 10 hyperlinks? I rarely see any threads started by actual forum users that have that many hyperlinks.

It's a bit of a stopgap, but should at least slow the current influx of spam threads we've been getting recently. Mod approval might be the best way. If someone REALLY wants to get a thread started with lots of hyperlinks, and needs it fast they can always send Brandon Martus or another mod a PM/email. Otherwise they remove the hyperlinks or wait.

z_beeblebrox 16-10-2013 18:42

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by Nuttyman54 (Post 1296863)
One thing I've noticed is that most of the spam has upwards of a dozen hyperlinks in it (most somewhere around 20). They also all seem to appear as new threads.

Would it be possible to put an additional level of security/captcha/mod approval type thing on any new threads that get started with more than 10 hyperlinks? I rarely see any threads started by actual forum users that have that many hyperlinks.

It's a bit of a stopgap, but should at least slow the current influx of spam threads we've been getting recently. Mod approval might be the best way. If someone REALLY wants to get a thread started with lots of hyperlinks, and needs it fast they can always send Brandon Martus or another mod a PM/email. Otherwise they remove the hyperlinks or wait.

This is a good idea, but in reality, a spammer often starts a thread and edits in the hyperlinks a few minutes later. That would be a little harder to deal with.

buildmaster5000 16-10-2013 19:15

Re: Spambot Prevention Suggestions
 
I thought that the forums had a 'you must have this many posts before you start a thread' rule. It seems that there are a lot of spam threads that are a first post by the user. Could something like 'you must post 10 times before you start a thread' rule be put in place? Or something that required mod screening then locking the thread for editing?

EricH 16-10-2013 19:50

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by buildmaster5000 (Post 1296869)
I thought that the forums had a 'you must have this many posts before you start a thread' rule. It seems that there are a lot of spam threads that are a first post by the user. Could something like 'you must post 10 times before you start a thread' rule be put in place? Or something that required mod screening then locking the thread for editing?

There isn't one, and I don't think there will ever be one. Most folks making a first post as a thread are trying to find help; even if there's a thread on that topic on the portal they might not know to look at it until AFTER their first thread.

But the bigger reason for not having a "X posts to start thread" is this: They'll (both spammers and non-spammers) just post all over OTHER threads looking for whatever they're looking for, often wildly offtopic--and the spammers do that already. So instead of seeing new thread with a lot of links pop up, you'll see an older thread pop up and think that "Oh, someone just posted, let's see what's up--Rats, spam" after opening the thread.

There is an "X posts to Y" rule, but it's 50 posts and positive reputation to be able to give non-neutral reputation.

Steven Donow 16-10-2013 19:56

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by z_beeblebrox (Post 1296866)
This is a good idea, but in reality, a spammer often starts a thread and edits in the hyperlinks a few minutes later. That would be a little harder to deal with.

Am I just crazy, or does the "this post was edited by..." text rarely show up when that happens?

Also, I was looking at Alexarankings today, looked up Chief Delphi, and some interesting results:

http://www.alexa.com/siteinfo/chiefdelphi.com

43.8% of CD visitors have an IP address in India.

Nuttyman54 16-10-2013 20:06

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by DevenStonow (Post 1296875)
Am I just crazy, or does the "this post was edited by..." text rarely show up when that happens?

That text only shows up after a certain amount of time has passed. I've gone back and immediately edited a post after I submitted it because I noticed something I wanted to change, and the "edited by...[date]" doesn't show up.

EricH 16-10-2013 20:11

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by Nuttyman54 (Post 1296876)
That text only shows up after a certain amount of time has passed. I've gone back and immediately edited a post after I submitted it because I noticed something I wanted to change, and the "edited by...[date]" doesn't show up.

It's not very long, though. Typically about 1-2 minutes, I think, from when you hit the post button to when you hit the edit button/save changes button.

magnets 16-10-2013 21:11

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by DevenStonow (Post 1296875)
Am I just crazy, or does the "this post was edited by..." text rarely show up when that happens?

Also, I was looking at Alexarankings today, looked up Chief Delphi, and some interesting results:

http://www.alexa.com/siteinfo/chiefdelphi.com

43.8% of CD visitors have an IP address in India.

Some of the other info there is really weird too.
The top three search terms are
1.delphi tableviewer
2.delphi forums (Ok, makes a little sense)
3.driver swag (????)
4.chief delphi (why is this number 4?)

yash101 17-10-2013 21:28

Re: Spambot Prevention Suggestions
 
That is cool.
OFF TOPIC: Also, i wonder where the web industry is going. My site running on a Pi has quite a high rank and is 50% speed rank.

Whippet 17-10-2013 21:44

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by magnets (Post 1296891)
Some of the other info there is really weird too.
The top three search terms are
1.delphi tableviewer
2.delphi forums (Ok, makes a little sense)
3.driver swag (????)
4.chief delphi (why is this number 4?)

I just went through 10 pages of search results for "Driver swag." I found nothing but lost hopes for humanity. All the others have this forum pop up on the first page.

*Spock eyebrow raise*
Fascinating.

MechEng83 22-10-2013 08:49

Re: Spambot Prevention Suggestions
 
Tangent: About half of the spam posts titles are mildly informative about world news events. If you're too immersed in robots, it's a good way to keep up to date on current events.

That being said, it's still a net detriment. (no pun intended)

yash101 23-10-2013 00:18

Re: Spambot Prevention Suggestions
 
Kind of off topic:
I placed a link to my forum (devstuff.no-ip.info/forum/) and said how wimpy it was (it runs on a raspberry pi). Apparently, the forum members are so helpful, they DoSed it! After reviewing the log files and other stuff, I found out that it was just because it was overloaded with requests. That gave me an idea of where I need to patch it up and now I am working on adding a maximum processes limit that will prevent apache from hogging all the resources from the kernel. The funny thing was that this time, the crash was so severe, even my UART connection was useless for degugging :). That made me do a hard reset :(. I do not think that that is a problem to the forum, but admins, you may want to place a resource cap to prevent the server from hogging all the resources, causing a kernel panic. I wish that there was a way to recover the logs from the kernel panic. However, that isn't safe as writing to the disk after the computer has crashed is VERY risky!

EricH 23-10-2013 01:17

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by yash101 (Post 1297959)
I do not think that that is a problem to the forum, but admins, you may want to place a resource cap to prevent the server from hogging all the resources, causing a kernel panic.

You aren't exactly familiar with CD history, so here's a couple of tidbits.

1) Legend has it that Brandon (the webmaster) used to put the CD server in a bulletproof case shortly before Kickoff, so that damage to surrounding things was minimal when it exploded. Not sure how true the exploding part is, but we'll go with that.

2) There was at one time a resource cap. Due to the amount of traffic, Brandon at one point locked the number of users online to a certain maximum--spiders were banned temporarily, guests had lower priority, registered users who were over the max couldn't get on for a while until someone else got off. Shortly afterward, the server got more space, so the limits were lifted.

And 3): The current solution is a bit of cloud-based server. Resources can be added in or removed to handle increased traffic (say, adding another couple of servers around Kickoff for the traffic, and letting them slide out in April or so). No issues other than the odd forum downtime, generally for an upgrade or to add in a server or three.

E Dawg 24-10-2013 12:49

Re: Spambot Prevention Suggestions
 
How about setting up a system where every post/edit MUST be reviewed by a moderator before being put up. It is easy to get past a filter, but people are much harder to trick.

JohnBoucher 24-10-2013 13:02

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by E Dawg (Post 1298200)
How about setting up a system where every post/edit MUST be reviewed by a moderator before being put up. It is easy to get past a filter, but people are much harder to trick.

The mods do a great job cleaning up the spam that gets through. They do not have the time to approve everything. When you report spam or questionable content, they get notified by email.

There needs to be a balance. I think it works here.

yash101 25-10-2013 22:22

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by E Dawg (Post 1298200)
How about setting up a system where every post/edit MUST be reviewed by a moderator before being put up. It is easy to get past a filter, but people are much harder to trick.

I think the webmasters have a life! Having to moderate every post will mean ChiefDelphi will need an army of moderators to approve the posts. Also, The posts won't show up until they are approved, which can be after something like possibly 24 hours! Instead, the first few posts should be moderated

Michael Hill 26-10-2013 10:39

Re: Spambot Prevention Suggestions
 
How about (when somebody registers), putting up a picture of someone like Dean Kamen or Woodie Flowers and asking for the person's last name. It's something that everyone involved in our community should know, but not so much outside. If they honestly don't know, make them register with admin approval.

yash101 26-10-2013 10:48

Re: Spambot Prevention Suggestions
 
Probably not the best thing to hear, but I don't know how these people look like even though I am on an FRC team. :yikes:

Michael Hill 26-10-2013 13:23

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by yash101 (Post 1298557)
Probably not the best thing to hear, but I don't know how these people look like even though I am on an FRC team. :yikes:

Then your registration would need an admin approval.

yash101 26-10-2013 18:38

Re: Spambot Prevention Suggestions
 
I'm pretty sure many people do not know how these people look. Most newbies won't know it either. maybe, we could have a captcha gif instead, with moving characters to make OCRs have a harder time!

Gregor 26-10-2013 21:27

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by yash101 (Post 1298589)
I'm pretty sure many people do not know how these people look. Most newbies won't know it either. maybe, we could have a captcha gif instead, with moving characters to make OCRs have a harder time!

I quick google search would show you their faces. You should try it sometime.

jamierose 27-10-2013 12:07

Re: Spambot Prevention Suggestions
 
Quote:

Originally Posted by Gregor (Post 1298603)
I quick google search would show you their faces. You should try it sometime.

I don't think that would work. If you're given a picture of some person you don't know, you'd have to do a reverse image search. If you do that, than anyone could do that easily.


My suggestion would be to change the captcha system to something like reCAPTCHA or Confident CAPTCHA, which John Fogarty suggested on page 3.

yash101 28-10-2013 10:25

Re: Spambot Prevention Suggestions
 
OK. Anyways, how would the moderators know whether it is a spambot or it is a actual person. What if the spammer creates the account by himself and allows the spambot to login off that account?
:deadhorse: :deadhorse: :deadhorse: :deadhorse: :deadhorse:

216Robochick288 28-10-2013 11:38

Re: Spambot Prevention Suggestions
 
I feel like the "multiple users report and it gets removed" is a good idea right about now. Seems like there is always a few threads around and about with people reporting them for spam, then it waits for someone else to remove it. Or again with the previously mentioned "Gets moved to an un-viewable thread to non-mods/admins who can either dump it or put it back up.

Is there a list somewhere off all mods/admins? I am kinda curious as to who of them are still active or not. Also, is there somewhere you can offer to be a mod? Perhaps another group could be made solely to remove spam if they see it. No more permissions other than moving threads to a spam folder for someone else to review and delete? How hard would these be to implement?

yash101 07-11-2013 23:03

Re: Spambot Prevention Suggestions
 
1 Attachment(s)
HELP!!! My random forum is being spammed with new members, all spambots. I can tell they are spambots because the email addresses are random!:eek: :eek: :eek: :eek: :eek:
Any suggestions?
I zoomed out! Here is a list.
somerandomuser and admin are the only valid users, as far as I know!

mechanical_robot 08-11-2013 08:29

Re: Spambot Prevention Suggestions
 
Do you think the spam is directly from a human or a spam bot. I mean I only see the spam threads once about everyday, it's not like they are flooding the whole entire forum so all we see is spam until we scroll down. Though what happens if the spammer turns out to be a person who is in the FIRST FRC competition themselves? What would we do then, I mean we have somebody who knows how these forums opperate and everything. Where I am going with this is that during the build season /competition season (January-April). Thats when these forums are going to be the most busy with threads and everything. So if you say this person (the spammer) was part of the FRC competitions themselves they would know the perfect time to flood this place with spam. If I was a spammer targeting Chief Delphi forums I would flood the place with spam during January-April when these forums would be busy with normal people trying to ask questions about their and other's robots and answer them.

So for the next few days I plan to:
  • Examine the spammers user anme
  • The email address they use
  • look for patterns (with their name and email)
  • Their team # they are using
  • location they do give

and more

Also if you notice the spammer is always using a different account
(because they have to ofcourse)

So in the next week or so I plan to have a report on the spammer about what I know and ways we can prevent it.

yash101 08-11-2013 15:02

Re: Spambot Prevention Suggestions
 
If someone wants a temporary mod account on my forum to see the types of spam coming, it may give a hint of what to look for on this forum. The email addresses are made up of random letters, gmail or hotmail! I think Google and Microsoft might like this to see what is happening with their email accounts!


All times are GMT -5. The time now is 01:35.

Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi