FIRST Choice- Why do we have this problem?

FIRST General Forum
#1

I realize that this post is going to sound like a big complaining rant, which is really what it is.

I am really disappointed, angry, and upset that the people who have set up FIRST choice are unable to make it work. Last year, I was angry that our team was unable to get any items that we needed, but I said OK, next year, they won’t screw up. But, they did. If andymark doesn’t have the infrastructure to support such a high load website then don’t use andymark.

Here’s my problem. Our team scrounged money together all summer long to pay for the expensive registration fee, so we expect to get our money’s worth. We were a little disappointed that we couldn’t log in last year because we had a low team number, but we just don’t have the resources and manpower to have somebody constantly refreshing a web site to get the first order in. We won’t be there when the people at andymark figure out what’s going on and restart the website, so we’ll get absolute crap like generator cords for our KoP. Meanwhile, the team that has a guy refreshing the website every 5 seconds will get exactly what they want. In my mind, that’s not fair.

The usual excuse of these people being “volunteers” doesn’t apply here. People have been paid to set up this miserable excuse for a web site, and our students have worked their butts of to pay to use this awful site. FIRST needs to reschedule the opening and give everybody a few days to prepare again.

#2

Are you sure?

#3

I was thinking of starting a new thread, but this one fits pretty nicely.

I’ve been thinking about FIRST Choice, and I’ve decided that if I were running it, I’d use a slightly different approach. IMPORTANT NOTE: This is not a knock against the folks at AndyMark, who do many great services to the FIRST community and who have worked very hard to put together a system that will stand up to a massive onslaught of page requests. That onslaught, in my mind, is the heart of the problem they are having right now. A system in which everyone has to sign on at the same time to get the “good stuff” is prone to these kinds of problems. So here is my approach (with apologies if someone else has posted the same ideas elsewhere on the forums).

When FIRST Choice opens up in November, teams are free to put their first order together during a two-three week open period. The order looks much like the current system, except 1) teams can order more than the number of points they have been allotted in case some of their items run out, and 2) the order includes a field for each item ordered that is the priority of the item for that team. Multiple quantities of items will be treated as separate items.

After the open period closes, the order system allocates the items in order of how many points each team has left. So at the start, when everyone has, for example, 150 points, the system goes through the inventory (in popularity order) and for each item it awards that item to teams that have that item as their first unfulfilled priority. If more teams want an item than are in inventory, a random pick decides who gets the item. At the end of round 1 every team has had their #1 priority addressed. Either they got it or not, On to round 2.

In round 2 the same algorithm is applied, only the field is a lot smaller, because only the teams with the highest number of points remaining are addressed. These are probably the teams that didn’t get their first priority in round 1 and still have 150 points left. All of these teams have their next unfulfilled priority addressed (i.e. their #2 priority item).

Each subsequent round goes like this until 1) all teams are out of points, or 2) all teams’ priority lists have been exhausted. (I thought that there was a #3 all inventory is gone, but that is covered by #2.)

At kickoff, the same method is applied, but the open period is much shorter, say until noon on the Monday after kickoff. Because teams completed their orders during the open periods, the shipping information will be in the system and ready to go. There isn’t any reason why teams couldn’t place orders after the open period is over, but those teams might miss out on the critical first rounds where the “good stuff” gets grabbed.

Have at it community, what are the flaws in my scheme?

#4

I think you should visit healthcare.gov. It might help your blood pressure. :slight_smile:

#5

Just throwing it out there:
FIRST Choice is a bonus that has been added (did not exist a few years ago) that generous donors are providing for FIRST teams. The parts are mostly surplus from previous years and should be thought of as a team donation.
Our registration fee doesn’t guarantee getting the best parts from FIRST Choice, nor does it help cover the PDV vouchers (except the AndMark one). These are all gracious corporate donations that we should be lucky to get. Registration pays for the teams to attend a Regional or District competition and receive the standard KoP.

I’m sure FIRST and AndyMark could have simply sold or threw away all these surplus parts, but they choose instead to hopefully distribute whatever’s left to as many teams as possible. They are effectively sponsoring the teams they give parts to and we should be happy to get whatever we’re given…

#6

I really can’t give AndyMark a hard time over this. Planning for huge loads is hard…like REALLY hard. Talk to some serious web developers about product launches and the unexpected loads that can be placed on a server. Some huge websites have crashed because of loads like this. I’m talking things like the Government’s Healthcare website, Target’s website and even The Olympic Games website. These are websites with IT departments 20 times the size of AndyMark. Cut them a break. The only thing you can really do in this case is to keep putting servers up (and hope you’ve optimized your code). I sound like a big enough cheerleader for Amazon AWS to my team already, but here goes again. I’d suggest hosting with Amazon EC2 so multiple instances can be spun up at any time (including product launches). It can get expensive, but it can definitely be worth it.

#7

Except it’s highly characterized… ~3000 teams can log on.

#8

I promise you that right now, AndyMark is probably just as angry as you might be. They strive to provide the best possible service to their costumers, and really do value their opinions. I’ve been fortunate to meet a few AndyMark employees, including Andy Baker himself, and I have nothing but good things to say about all of them. They are some of the most passionate, enthusiastic, and kind-hearted people I know, and I’m sure they are extremely upset about the problems that occurred as well. And the same goes for everyone that works (and volunteers) at FIRST. But it’s all a learning experience. It’s not like what they are doing is easy. They’ll see what went wrong, and learn how to fix it for future years. I challenge you to do their jobs better. Tell us how that goes.

But isn’t that what *FIRST *is about, learning how to deal with the unexpected problems that are thrown at you? :wink:

And, if the biggest problem in your life right now is seriously just a website not working, well, then you’ve got a pretty good life!

Sit back, take a breather, and just enjoy the bigger picture of life for a bit.

#9

maybe, just maybe they came up against a problem they didn’t expect or was out of their control?

On a slightly different note: We are all in the same boat. No one is getting an advantage because of this issue. Let’s all relax, watch some holiday TV and get ready to log in @ 3pm eastern. Some of you might want to go to the early church service :wink:

#10

Agreed. I would think the cost of spinning up a bunch of instances during the times when you would know there would be heavy demand (such as the first 3-4 hours today, then the 3-4 hours in January), would definitely be worth it. Another thought is to try and use Cloudflare, to help cache some of the files, reducing the load on the servers. I implemented it for my old team’s site, and it seemed to work well (although we never had the demand like FC does ;).

#11

I’ll take this over last year’s mess any day.

No ones getting swindled out of talons because of this, tomorrow is a fresh start, relax.

#12

Let’s cut AM a bit of slack, folks. As one of the few people on CD who has actually run big sites, it’s not as simple as “spin up a couple more EC2/web role instances and you’re golden.”

Building a transactional ecommerce site that goes from zero utilization to a peak that lasts only a couple hours a year is a tricky endeavor, particularly a big investment in testing. This site probably takes a load similar to a Ticketmaster ticket sale for a popular 20000 seat arena show.

They clearly put much more effort this year into making it work, and ran into an unforeseen problem. They know it’s broken. Let’s let 'em fix it.

#13

As far as issues go, this is relatively minor one. I can see where that the opening being rescheduled might be problematic for some people that had planned to be on today, but overall I doubt it’s much more than the teams who weren’t going to make it immediately anyway.

As far as the actual website goes, I find the actual website to be rather well structured. Obviously right now it’s fulfilling the main requirement of, you know, being up, but the pages themselves seemed good when I was looking at it earlier this month.

To be honest, I don’t really see how someone could call FIRSTChoice a “miserable excuse for a website” when it literally solely exists to give us all free stuff. Maybe there’s something of a competitive advantage to getting a free talon or two, but it’s not like you can get an early copy of the game manual or something on there.

#14

I will say to AndyMark’s credit, they haven’t shirked their responsibility or tried to blame others. Andy Baker’s post makes that clear. FIRST’s official blog post about the incident and reschedule implies that the service provider had something to do with it.

They’ll get it fixed.

#15

Unlike last year’s debacle, this was correctly handled and no big deal. It will be a scheduling inconvenience for some, and good news for others. A one day delay for a do-over is vastly better than allowing technical problems to cause unfair results. I expect it to work tomorrow.

#16

I’ve heard that more than a few times in my life. :slight_smile: It’s no big deal. We’ll get to spend out 150 points sooner or later.

#17

Agreed. So do I.

We have worked on many things today, and have the full attention of our web services provider. We are conducting one last test, while we have a chance:

If folks are able to poke around the FIRST Choice site tomorrow (Sunday) at 11am Eastern time to try to overload the site, it will help us determine if our findings today are correct. So, if you wish to kick the tires some more with us, please visit the site for a few minutes at 11am.

Thanks,
Andy

#18

Perhaps, but according to the Andy Baker thread, Andy himself can handle 38,000 simultaneous connections. I heard that when they realized the server failed, they put Andy in front of a computer to handle all the orders manually…but the keyboard melted…

#19

It’s not free, and it’s not guaranteed. But it’s interesting to observe how framing it like a bonus changes (some) people’s expectations.

As part of your registration, you paid for the chance to receive certain things. Treat it like the mandatory purchase of lottery ticket: from the entrant’s perspective, it’s not particularly deterministic. If you come in with that (modest) expectation, it’s hard to feel cheated. (As a corollary, it’s hard to be particularly excited about it either.)

Having said that, I suspect that making it transparently deterministic (except for breaking ties) would be a good idea. There are a limited number of items, and a limited number of teams, so doing the computations in several instant rounds is no big deal. Teams could select items, quantities and preferences (e.g. as preference points expressed as decimal fractions summing to 1). When it’s time for the first round, it attempts to allocate teams’ highest preferences (in order of most preference points expended). If there are teams who don’t get their first preference, the teams’ remaining preferences are normalized to fractions summing to 1. (So if you miss out in the first round, your preferences are strengthened in subsequent rounds.) Then the next round instantly takes place, and so on. I haven’t thought this through in detail, but it seems to me that even if there are edge cases, they can mostly be rectified by offering teams variable-value PDVs and/or less-than-par fee rebates (proportional to the amount of unfulfilled preference at the end of all allocations). After all, even if you get no parts, it’s harder to complain about a discount.

(I haven’t reviewed the FIRST Choice allocation algorithm lately, so perhaps some of this is in place this year.)

#20

I feel like I have to apologize for my earlier post in this thread. I posted it after I though our team would be cheated out of a fair chance for parts for two years in a row. However, I now feel like a total idiot to all of the people I just insulted. I do realize that things can happen that are completely out of the control of FIRST and AndyMark, like what did happen, and I really think that FIRST did an awesome job of responding. Any previous orders were canceled, and all teams get to have a fresh start.