Hobbes directory structure re-organization

[Martin Iturbide]

Hi.

I'm proposing a file structure reorganization at hobbes. I will like to have some discussion on the forum of what can be improved on the hobbes directory structure to store files.

I currently made a draft on Freemind about the structure based on the current one that hobbes use. I'm sharing this files for you to comment.  Fell free to modify it and present any option, or give any comments.

Regards
Martín

[Doug Bissett]

Hi.

I'm proposing a file structure reorganization at hobbes. I will like to have some discussion on the forum of what can be improved on the hobbes directory structure to store files.

I currently made a draft on Freemind about the structure based on the current one that hobbes use. I'm sharing this files for you to comment.  Fell free to modify it and present any option, or give any comments.

Regards
Martín

I doubt if your effort will be rewarded. You must remember that HOBBES is managed by a student, who is learning how to do such things. They rarely have enough time to manage what exists, never mind try changing it in more than the most minor ways. It appears that nobody else does much, if anything, with HOBBES, and a student isn't likely to learn enough, in 4 months, to be able to make large changes, without major disruptions. Plus, they need time to go to classes.

Then, there is the problem where there are thousands of posts, all over the internet, that link to files where they exist today. Changing HOBBES would likely break a lot of those links.

FWIW, I think that HOBBES is not well organized, but most of the problem is caused by those who upload things to inappropriate places. Reorganizing things, isn't going to fix that problem. It is, after all, a user operated service, with a little help from somebody at NMSU. It is usually pretty easy to do a search to find what you are looking for (as long as it has been properly described by the uploader), so there doesn't seem to be a pressing need for changes to HOBBES itself. However, if you want to encourage users to make more logical decisions about where to put things, and how to properly package (and describe) what they upload, the problem may be fixed, eventually. The other thing that could help, is to document all of the obsolete software that is at HOBBES, and see about getting rid of some of the overhead. Of course, as soon as something is removed, somebody will be looking for it.

Overall, I am very impressed with the way that HOBBES works, today. It is certainly not perfect, but it is better than most file repositories.

[Dave Yeo]

I have to agree with Doug, and will add that the Hobbes today is much better then in the past. Some examples,
Here is an example from 1992, http://stuff.mit.edu/afs/sipb/project/os2/ftp/. Today it would probably be the develop subdirectory, all programs are archived with zoo, newest GCC is 2.2.2, much won't even run without recompiling or LIBPATHSTRICT along with the ancient EMX dll. In many ways a mess but interesting.
A couple of years later and it had changed quite a bit and the old stuff was gone, http://cd.textfiles.com/hobbesos2/. Everything there has been lost due to a re-organization and that is another problem with suggesting re-organization, be horrible if some student wiped it to start over.
More interesting CDs including a few more Hobbes archives at http://cd.textfiles.com/directory.html. Shame they don't have a LEO archive as that was the other large OS/2 repository before they lost everything in a crash 

[Martin Iturbide]

Hi

I understand your points, but I was offered an opportunity to improve hobbes and I think I must try at least to take the opportunity.
 
I take notice of the points:

1) What about the sites linking hobbes files?
2) We must encourage the file contributors to upload the files in the right places.
3) We need more active administration/moderation of the site. (to organize files)
4) Search is a very important service of the site.
5) If a migration/update happens, don't lost any files.

Doug, do you know the skills of the people involved at hobbes today? or you are just telling about your experience some time ago. I don't think we must disqualify nobody, specially when it is a community effort and nobody is paying hobbes to be online. 

My general question remains open for everyone. Do you want to help (or at least try) to reorganize hobbes directory structure to have a more modern site? Please check the structure I created and give me feedback about the structure.

Regards

[Pete]

Hi Guys

Probably the most useful change would be getting hobbes internal linking to work.

The classic example is the link Return to homepage at the bottom of each webpage which does *not* return to homepage - http://hobbes.nmsu.edu/ - but instead reloads whatever page you are currently viewing. Some example urls that have this problem (it is the same problem on every page on the website I've visited):-

http://hobbes.nmsu.edu/h-browse.php?dir=/pub/os2
http://hobbes.nmsu.edu/h-browse.php?dir=/pub/os2/games
http://hobbes.nmsu.edu/h-browse.php?dir=/pub/incoming&sort=date


Martins structure looks OK to me - but I claim no expertise in these matters.


Regards

Pete

[Ian B Manners]

FYI,

I do have a complete FTP mirror of hobbe's that is updated every month,
I also use to FTP mirror leo, so I have an almost complete copy, I just cant vouch for any files put on the month before it vanished.

I don't make these directories available via os2site.com simply because all the files are already integrated into the os2site.com/sw/
directory structure but I can probably add them for OS/2's access if there is interest now that I've greatly reduced the load on my
home server.

[Ian B Manners]

Hi Martin,

os2site's directory structure is :-

http://www.os2site.com/0.dir.list  if you or anyone else would like to compare or look at for ideas.

I dislike deleting older files so tend to move them to an 'old' sub directory, and yes, moving file URL's is an issue but many people
will simply go back to root and work up a new tree, or use hobbes search facility. os2site.com though does still get hits for old
links for files from 2000...

Occasionally I will get a request to remove a file, in which case they are moved to my swc section so they are not publicly available

os2site's file structure started off identical to hobbes, then mutated when I assimilated Leo, and like all living things, it's evolved over
the past 13 odd years.

I'll go over your structure file shortly.

On another note, the internal links issue with hobbes wasn't picked up on the test site so it wasn't listed to be fixed, apologies, I missed that one but now we have the option to say something

[Doug Bissett]

Doug, do you know the skills of the people involved at hobbes today? or you are just telling about your experience some time ago. I don't think we must disqualify nobody, specially when it is a community effort and nobody is paying hobbes to be online. 

No, I don't know, for sure, but it is pretty obvious that at each semester change, there is a new person learning the ropes. Sometimes, HOBBES goes for a whole semester with nothing happening. I expect that there is an overall supervisor who makes sure that nothing too serious happens. The actual person in charge is probably the guy who asked for contributions, to upgrade the system, a few years ago.

Your structure looks okay, but there are probably a lot of different ways to do it, and all of them would be equally acceptable. Something that follows the current setup would probably be the easiest to implement. Some cleanup would help more.

Shame they don't have a LEO archive as that was the other large OS/2 repository before they lost everything in a crash

Hopefully, HOBBES has a good backup plan in place. Since HOBBES itself is only a small part of the whole site, I expect that it is well looked after. It is reassuring that there are a couple of mirror sites.

The classic example is the link Return to homepage at the bottom of each webpage which does *not* return to homepage

Hmmm. I never noticed that, since I never use it. Yes, things like that should be fixed.

I can probably add them for OS/2's access if there is interest

Ian, you have a great site, but general access is a bit awkward, although not difficult. I think your site could be used as a template for any changes at HOBBES, but I still question whether such changes are actually necessary (some cleanup would be nice).

While we discuss backups, and mirrors, I assume that you have a good backup plan in place, but what happens if  a small asteroid clobbers your residence? Is there anyone mirroring your site? (I know it is "forbidden", but exceptions can be made). It would be a shame to lose your site too.

[Ian B Manners]

Hi Doug,

Ian, you have a great site, but general access is a bit awkward, although not difficult. I think your site could be used as a template for any changes at HOBBES, but I still question whether such changes are actually necessary (some cleanup would be nice).

It's awkward on purpose unfortunately.

The simplicity of hobbes is its beauty and as much as I wish to say add things like directories to store older versions to tidy up current directories, I'm worried that will mean a human needs to look after it, that's the hard bit, getting someone who understands the software, our naming convention etc. We could send them a list of files once a year or blue moon of files to archive off, possibly.

While we discuss backups, and mirrors, I assume that you have a good backup plan in place, but what happens if  a small asteroid clobbers your residence? Is there anyone mirroring your site? (I know it is "forbidden", but exceptions can be made). It would be a shame to lose your site too.

The executor of my will is on the other side of Australia, as is my backup server I left in Melbourne that mirrors my main server every week.

In the event of my death (possibly by asteroid strike) then a short 'item' will be posted on os2world.com if still running, and in the OS/2 newsgroups as well as a couple of other places asking for expressions of interest in becoming the new owner of the os2site.com server(s), if both servers survive then the preference is that each one goes to individuals/entities at different locals.

The Australian National Library also has an almost complete copy of os2site.com and all my Digital Equipment Corporation software and information.

By the time the post is made, any of my current users would have copies of there data from the server, and that data removed.

There are also rotating local backups to 1TB USB drives where I am currently located, and two spare servers which are DSync'ed when I remember to do so.

If both servers and all backups are destroyed, I think anyone else in the world that survives will have other things to worry about )

Bit overboard I know, just something that's evolved.

[edited for speeling]

Others can mirror os2site.com but it would have to start with me sending them DVD's to preload, otherwise that's a lot of data to download depending on if they want all the IBM ftp stuff as well.

[Added as an idea]

[Doug Bissett]

We could send them a list of files once a year or blue moon of files to archive off, possibly.

Hopefully, it won't be necessary to use such crude methods. 

Bit overboard I know, just something that's evolved.

It seems that you have the situation well under control. May you, and your servers, live a long and happy life. 

Others can mirror os2site.com

It might be a good idea to have a mirror outside of Australia (no, I am not volunteering, at the moment), but I can't think of any really good reason to do that, other than diversity.

and two spare servers which are DSync'ed when I remember to do so.

Could I suggest changing that to an RSYNC copy operation? I used DSync for a while, and it worked well, but RSync does just as well, and it is a LOT faster, mostly because it doesn't just copy the files. It does a smart copy, which reduces the amount of data to be handled (after the first time, of course). Currently I am handling about 10 times the amount of data, in half the time that DSync required. I use it for my nightly backups to my USB drive, as well as to my home built NAS box (eCS 2.1) across my Gbit network. If you do something through the internet, it can also ZIP data for transmission, if that helps (it depends on the speed of the link).

[Ian B Manners]

Hi Doug,

Could I suggest changing that to an RSYNC copy operation? I used DSync for a while, and it worked well, but RSync does just as well, and it is a LOT faster, mostly because it doesn't just copy the files. It does a smart copy, which reduces the amount of data to be handled (after the first time, of course). Currently I am handling about 10 times the amount of data, in half the time that DSync required. I use it for my nightly backups to my USB drive, as well as to my home built NAS box (eCS 2.1) across my Gbit network. If you do something through the internet, it can also ZIP data for transmission, if that helps (it depends on the speed of the link).

I kept getting the odd error message with RSync so stayed with DSync (A few years since I tried it). One day I'll change, just not on my high priority list at present, and its only used locally so it's quick enough.
The Melbourne server does a simply wget -m operation on the data drive only, so Rsync isnt likely to make much difference data transfer wise as it's copying new, not altered files apart from the odd description and index.html file.

Now back to hobbes

[Martin Iturbide]

Hi Ian

I updated the proposed structure based on something I was able to see on OS2Site.com/sw.

Let's keep discussing it.

[Martin Iturbide]

I was wondering, can we dare to write a "Hobbes File Naming Suggestion" document?

For example:

You can name the file after your product and indicate the version of it:

Ex: For an application named "Warpinator" release 1.5 you can name the file "Warpinator_1-5.zip"

Ex: What not to do. Do not refer your file as "latest" like "Warpinator_latest.zip"

It is a suggestion, it will not be written on stone of course.

[Doug Bissett]

I was wondering, can we dare to write a "Hobbes File Naming Suggestion" document?

For example:

You can name the file after your product and indicate the version of it:

Ex: For an application named "Warpinator" release 1.5 you can name the file "Warpinator_1-5.zip"

Ex: What not to do. Do not refer your file as "latest" like "Warpinator_latest.zip"

It is a suggestion, it will not be written on stone of course.

It seems that there are certain file extensions that are acceptable, for uploads, and others that are not. This is a good thing, but some users don't seem to know what is acceptable.

For instance, a number of packages are built as WarpIN installers, which are then zipped. This is completely redundant, and often (if not always) results in a larger file for upload/download, plus the user needs to take the extra step to unzip the files. Files with the extension WPI are acceptable to HOBBES, and should be used. I have never tried it, but I believe that EXE files are also acceptable, for self extracting files (but that should be verified).

It also seems that HOBBES prefers (sometimes demands, depending on the maintainer at the time), that file names should always be lower case. OS2/eCS users often forget about that, as you did in your first example. It would be good, if HOBBES would ignore the case of the file names, but that is up to them, not us (won't hurt to ask).

I am sure that there are many more things that should be addressed, while you are at it. As I have said before, most of the "problems" with HOBBES are caused by the users. A comprehensive user guide might be more useful than reorganizing the whole thing.

[Martin Iturbide]

I just remember that there are some EXE that I don't consider acceptable anymore. There used to be some IBM's EXEs that unpacked the files into a diskette drive, on these days where we don't longer have diskette drives and those EXE are unpractical. I remember that I found some sound drivers with that EXE installer and I re-pack them as .zip and upload them to hobbes.