File Sharing and Revision Tracking

[Madison]

As I was going through trying to imagine how the interface for freelanceFIRST would look and operate, I began by examining the features of each component of the site, their hierarchy, and the potential frequency of their use.

When I came upon the project file exchange, a part of the project environment that allows users to exchange project files, I ran into some trouble.

What is the best way to successfully implement a file sharing system that has revision tracking?

That is, we wouldn't want a user to accidentally overwrite a file on the server with an older, or incorrect, version. There needs to be some method of tracking who has uploaded and downloaded project files, when they've done so, and what changes they may have made.

Is it wise to 'check-out' a file, much like a library book? So, for as long as an individual has a file, it's unavailable to others?

Or, maybe 'control' of the file could be given to a single user at a time, but others could still use that file simultaneously. Any changes made by someone other than the user in 'control' would result in a duplicate file being created on the server. With this system, someone who's not in control can offer suggestions or make changes, and do so without fear of interrupting the work flow of the project.

Inventor files seem like they'll be a particular challenge. While Inventor assembly files update to reflect the status of the parts therein, if major changes are made to a part, it'd probably require that the assembly file be updated as well.

Likewise with Inventor project files. Maybe we could require that a new Project and a new Assembly be uploaded with each revised part? That seems a little ridiculous, and probably unnecessary.

I'm open to any suggestions on how to implement this feature. What features would you look for in managing a project? What would you like to know about it as a user?

I want this to be as intuitive and friendly to use as possible. I'm not a big computer geek, so I want to be able to use it without frustrating myself.

Think of this as something similar to the game challenge we receive every year. You're each faced with the task of designing this system. Let's see what we can do to find the most effective, efficient, reliable solution possible.

[mtaman02]

you could try using an FTP type program it allows users to download files and users may upload files. I don't know the theory behind how it works or if theres a revision tracking program on it at all.

Basically all it requests is these things

ftp.whatevertheserveris.com ---> i'm assuming
username=user
password=12345 (etc...) and from there someone such as myself can see what files is on the server upload / update what ever I can put there and I can download whatever it lets me download.


i think brandon knows alot more about this than I do since chiefdelphi works kinda the way you are describing in the post. So Brandon take it away!!

[Brandon Martus]

Keeping a record of revisions is tricky... especially with binary files.
For ascii, you can just keep a record of the diff , along with the newer file. For binary, if you want to keep track of revisions, you may need to keep those older files, which will start to take up alot of harddrive space.

As for locking files, that may be necessary. We use cvs for version control at work, and the only way to check in changes is to have the most recent revision + your changes. If someone updates the file while you are changing it, you must update yours before checking in your changes.

You could put temporary locks on the files, and send e-mail reminders to the user to either renew the lock, or let the lock expire (or force-expire it, if they are not going to need it anymore). You could also put a system of lock requests into place, where people could request the next lock if the file they need is currently locked. That is something you'd need to work out in detail, though. There are alot of different scenarios that could happen.

[Josh Hambright]

I dont know how well this would work over the internet but i remeber reading a review on webmonkey.com about Macromedia Contribute which i think is for use like on LAN's for users to share their work and develope together and it has alot of features that your looking for.

Here is the official site:
http://www.macromedia.com/software/contribute/

Here is the webmonkey review:
http://hotwired.lycos.com/webmonkey/...l?tw=authoring

This is meant more for images and HTML files i think which could help somewhat. But i have no idea on how to do it for inventor files. We actualy had some sharing and revision problesm with the whole team working on inventor this year, eventualy we just had to assign people to be incharge of some of the bigger more complex assemblies...and you had to ask around before you made a change to make sure nobody was using it.

[jon]

My first thought was CVS, as Brandon ?sugested? Anyway, I wasn't quite clear on how CVS dealt with binary files... so I did a little googling and found this . It basically says what's already been stated.

Text is no problem, but for binary, you can't always have a diff. So storing all the binaries seems the best way to go. Now I'm not saying freelanceFIRST won't be big, but I don't think storage will be much of a problem. I mean, there wont be THAT many projects. And as it's popularity increases over the years, storage space will go down. See, it all works out great.

This quote from the manual basically states how I think update should be handled.

"Another ability of a version control system is the ability to merge two revisions. For CVS this happens in two contexts. The first is when users make changes in separate working directories (see section 10. Multiple developers). The second is when one merges explicitly with the `update -j' command (see section 5. Branching and merging)."

People make their changes, and the main file is updated when everyone agrees that it's great.

[Brandon Martus]

Originally posted by jon
People make their changes, and the main file is updated when everyone agrees that it's great.

FYI: That can get very .. 'icky' (for lack of a better word) .. when working in the same areas of a file. CVS conflicts can get very nasty.

[Madison]

Okay, so.

Each file has certain relevant characteristics associated with it.

They are;

• file name - this should be created by the site based on given information. i.e., "InventorFile(Jon060303).idw" or something. This will prevent overwriting important files on a person's hard drive.
• file extension - likely included in the file name. it otherwise wouldn't need to be displayed.
• file size - self-explanatory, I'd guess.
• description - a brief description of what the file is or contains.
• date created - the date the file was created/uploaded. Is there a difference? Should there be?
• last revision - The date of the most recent "promotion." for the file. I'll get to what a promotion is in a bit.
• created by - The user that created the file originally.
• revised by - The last user who's revision was "promoted."
• checked out by - which user has checked the file out for use.
• due back by - when to expect that user to be finished and the file to be checked back in.
• project association - which project the file belongs to. this doesn't need to be displayed.
• subgroup association - which subgroup within a project the file belongs to. this doesn't need to be displayed.

With these characteristics, users and administrators should be able to learn as much about a file as they'll need to know.

There needs to be some file tracking and a hierarchy. For this, there are five primary functions, and within each of those, there may be secondary functions.

• download - download selected file or files. here, a user is given the option to "check out" one or more of those files, indicating they'll be making changes to them and causing their username to appear in the initial file information as outlined above. "checking out" a file requires that a user decide to lock the file or not, prohibiting further downloads, as well as that they indicate when they expect to return the file. If a file is currently checked out, a message will indicate this, or that said file is "locked."
• upload - upload files from a users offline or online storage device. here, users can choose to "check in" a file if they currently have it checked out. if a file is "checked in" it overwrite the master file. if the file isn't checked in, or is being uploaded by someone without possession of that file, a "pending" group is created below the master file.
• promote - used to change the status of a "pending" revision to the master file. this will overwrite the master file.
• demote - used to change the status of a master file to "pending." this may result in there being no master file. this is useful because a group may want to keep the old master file on hand.
• delete - delete a file from the exchange

This methodology combines locking a file from use and keeping all revisions and changes available. By giving a user the ability to check out a file, it provides a way to indicate that changes are being made, but it doesn't prevent others from examining the files on their own, and possibly suggesting changes. It's likely best that certain functions, such as a promote, demote, and delete are only available to certain users. My user control panel features a similar promotion and demotion system.

Hopefully, by creating a list of pending changes to a file or files, it'll maintain the flexibility of multiple users working on the same project and file, but it'll still offer some degree of control over the changes, as well as preserve a bit more bandwidth and storage space.

How's that?

[Jack]

ok.. this replys to really both this and the other thread. (website & gui)

wow.. The ideas you've created are awesome..

All I can say is that they're going to require a ton of codeing to be done.

If you all decide to use php, I'll be happy to help out. Since you will be having a forum, using that as a backend for templateing, db interaction, user accounts would reduce the work load a lot. However, it would then require all of the coders to be able to understand the workings of that type of system.

In reply to your last post, you're definatly going to need some type of custom system to handle that. (And cvs won't begin to work for inventor files, etc...)

Also, In terms of hosting. Well.. the FSN project is going to be getting a ton of web hosting (cross-fingers). In the least, one dedicated server and another VPS. (Yes, I write very good begging emails ) Our hosts really don't care whether the hosting would be used for fsn or flf (freelancefirst). - As long as they both are for educational things and all hosted pages contain a link back to the sponsor at the bottem. Once I get the things set up, and see how much we actually will have, I'm sure we could work something out. I'll get back to you all on this

Jack

[Madison]

Heh. See, the thing is, I kinda remember HTML. That's it.

I'm completely relying on the wisdom and guidance of other people where coding this monster is concerned. I've had suggestions for PHP and Flash. I don't know the difference, really, except that PHP is likely all words and semi-colons, and Flash uses pictures. But, I'm sure there's more to Flash than that.

I just keeping tossing things out there, and relying on other people to make suggestions about how it might be implemented, if it can be implemented at all.

I think my next project should be equally monstrous "Thank You" page.

[Jack]

Originally posted by M. Krass
I've had suggestions for PHP and Flash. I don't know the difference, really, except that PHP is likely all words and semi-colons, and Flash uses pictures.

hehe.. well yes.. you got that corect!

If you think that flash and php are exclusive, they aren't. Let me give you (and others who don't know) a two sentence overview of the two.

PHP: This is what is known as a server side script. If you notice, all the pages here on cd have a .php for the the file name. What php allows you to do is generate dynamic content. It can query info from a database, handle files, and do a million other things. In the end, the php file will output the final source code to the browser - this is what the end users sees.

Flash: This is the "front end" of a site. While parts can be dynamic (say a menu bar using a xml file for it's buttions), it's nothing like php. Personally, a little flash is nice. But, an enitre flash site is harder to maintain, requires a fast connection, and won't work in some older browsers. You're still going to want most of your site to be html.

My $.02

[mtrawls]

Let me add another comment on flash. I don't recommend using it. Why? Well, as was said, it requires a fast connection to really work as developed usually. I don't think that the site would want to alienate people without a cable internet connection. As said, it doesn't always work with older browsers. A lot of the internet has the latest browsers, sure; but a lot don't! Those users shouldn't be alienated. Not everyone has Flash plugins. While PHP is server side, Flash is client side. That means that instead of relying on ONE computer to have the necessary support, you rely on EVERY computer to have the necessary support. So either they get Flash or they can't see something on the site. That can be aggravating, and can alienate even more users.

But one of the biggest problems I have with it is that people use Flash a lot of times to give visual appeal to their websites. Okay, that's nice ... but then they never work on the actual content -- which, if you think about it, is why the user is there in the first place!

I don't mean to go off on a usability rant here, but I think it would be a real shame if this site was designed for only a small portion of the internet. It will be important to keep in mind that some people might not be using IE6 or newest/latest browser from Microsoft with every plugin. Not everyone has a T1 line (okay, slight exageration, but I suspect many people may be going there from school ... and my school at least has notoriously slow connection speeds), so massive images or flash files, or anything requiring excesive times to load is not good. People might be acessing the webpage in a text only browser. Or even from a handheld device. They might have dissabilities (see the W3C guidelines for this)

On website design, I think the biggest mistake is that people forget to keep things simple. Webmasters forget that a user is there to get whatever information his site has to offer and not to see the latest new-fangled internet wizardry available. Sure an animation might be "cool" or "neat" ... the real question is, does it somehow explain or relate information to the user better and/or more succintly than text, or some other thing.

[Nate Smith]

Originally posted by Jack
ok.. this replys to really both this and the other thread. (website & gui)

If you all decide to use php, I'll be happy to help out. Since you will be having a forum, using that as a backend for templateing, db interaction, user accounts would reduce the work load a lot. However, it would then require all of the coders to be able to understand the workings of that type of system.

First off, here's the URL to what I've been playing around with so far:
http://freelancefirst.pitnews.org/forums/
As you can see, it's based around the Invision Power Board forum software. From my experience, it's not that bad for skinning(I've worked a little bit on this already) or writing mods for.

Also, I've been toying around with how to handle the version management system myself. Here's what I came up with:

-User A wants to use a file, so they check it out and download the current version.

-In the checkout tracking DB, the following is recorded(english language translation follows): "user A checked out file XXX.BSX version 1.5.2 on $date at $time"

-While User A has the file, user B wants to use the same file. When user B goes to download the file, they get a warning stating that user A is already working on that file, and asks them if they still want to download the file.

-If user B decides to download the file, the following is recorded in the checkout DB: "user B checked out file XXX.BSX version 1.5.2 on $date at $time"

-When User A or B checks back in their file with whatever modifications, a branch is created under the checked out version for EACH returned file.

-The project owner or project leader(owner=who requests it, leader=who accepted it) is responsible for deciding which branch, or a combination of the two, gets the 1.5.3 version identifier. The file(s) which do not recieve the new version number get tagged as "interim versions"(unless I can come up with a better name)

What do you think?

[Jack]

Not a bad idea nate.

Of course this leads to some people's work begin wasted, but that would occur anyways if the first design wasn't liked.

[Brandon Martus]

Originally posted by Jack
Not a bad idea nate. Of course this leads to some people's work begin wasted, but that would occur anyways if the first design wasn't liked.

They would be warned, though, so they'd be knowingly wasting their own time.

[Madison]

Originally posted by Brandon Martus
They would be warned, though, so they'd be knowingly wasting their own time.

It also gives the project leadership an opportunity to see two or more different solutions to a problem.

While one person may have control over a file, another may see a unique solution to a problem, and should have access to that file.

Sounds good to me.