OS/2, eCS & ArcaOS - Technical > Storage
Finding an ideal filesystem for Network storage
Rich442:
In terms of managing an OS/2 network of storage devices, what would be the most effective filesystem for speed and efficiency? I really like the way ZFS works on UNIX and UNIX-like systems (combining the volume manager with the actual filesystem). Most of my storage devices are on the small size (i.e. 6 hard drives of about 120-1000GB per disk drive) and I have ecomstation 2.2 on my main device. Since HPFS and JFS both use the B+ tree, what would anyone recommend to manage a network of six storage devices of : 120GB, 60GB, 300GB, 1TB sizes.
My goal is efficient data retrieval where data is preserved (in case one of the disks fails). How does HPFS handle the problem of disk failure and data corruption? My SATA disks are relatively old. I'm really interested in the benefits of HPFS over some of the NT filesystems made by MS. Thanks for any responses. :)
Dave Yeo:
For OS/2, it is basically JFS for modern large hard drives. HPFS is limited to 64GB partitions and takes forever to do a chkdsk on a large partition due to no journaling.
Rich442:
Thank you for responding. I do use JFS on big partitions but my "network" consists of a lot of different oddly-sized stuff from various. A few are 20 years old and the rest are at least ten years old. One of my most reliable drives has only 60GB on it so I really wondered. It was sold by apple and I usually use HFS+ on my Mac stuff.
My real struggle is whether use JFS rather than FAT32 (Windows) or HFS+ for this conglomeration of old and older disks. I referenced ZFS (from Oracle because I thought it would be an example of how best to what I want to do: speed, ,managing devices and preventing loss of data/data corruption. I was hoping that JFS had something comparable that would make my network faster and safer (ZFS being an example). I like a b-tree structure but also like the way FAT32.
Would it be better to simply run regular backups with something like a cron script?
RickCHodgin:
--- Quote from: Richard Crowley on December 13, 2017, 12:40:00 am ---In terms of managing an OS/2 network of storage devices, what would be the most effective filesystem for speed and efficiency? I really like the way ZFS works on UNIX and UNIX-like systems (combining the volume manager with the actual filesystem). Most of my storage devices are on the small size (i.e. 6 hard drives of about 120-1000GB per disk drive) and I have ecomstation 2.2 on my main device. Since HPFS and JFS both use the B+ tree, what would anyone recommend to manage a network of six storage devices of : 120GB, 60GB, 300GB, 1TB sizes.
--- End quote ---
There are two aspects. First, there should be an aggregating server setup which coordinates which volumes are online at any given time. That is a manager and is queried by any server wishing to populate data onto the network drive, or any client machines retrieve information about the online devices. And a single OS/2 instance could be both a server and a client.
That aggregating server maintains a virtual map of current online storage, which is then transmitted to each client machine requesting network drive access, with the aggregating server sending out push notifications whenever resources change.
In this way, each server registers its own public data with the aggregating server, which is all aggregated into a single central directory, with the individual resources being indicated that they are on physical servers, which each client then directly communicates with for services on those files on that machine.
This is for data I/O. The file system then in use would be of any kind OS/2 supports, possibly with emulation to allow OS/2 attributes on a non-OS/2 file system. But as a file system of choice, I would suggest JFS in moving forward.
I think any modern networking file system has to be both distributed and aggregated as indicated above. The traffic to the aggregate server would be minimal, and it would constantly communicate with each online resource and signal when things are reliable, unreliable, offline, online, etc., to all client machines connected to it. It could also perform mirroring and management from a single source, directing files to be moved from one machine to another, copied, all from a single console.
In this way, a single "network volume" is made visible with essentially unlimited storage, with the physical requests to each of the network resources going out to the specific machines to be filled.
--- Quote from: Richard Crowley on December 13, 2017, 12:40:00 am ---My goal is efficient data retrieval where data is preserved (in case one of the disks fails). How does HPFS handle the problem of disk failure and data corruption? My SATA disks are relatively old. I'm really interested in the benefits of HPFS over some of the NT filesystems made by MS. Thanks for any responses. :)
--- End quote ---
HPFS was originally created by Microsoft. As I understand it, NTFS was basically a full-on fork of HPFS at a given time, examined, revised, and re-written for Microsoft's purposes thereafter.
UPDATE: I have not seen this kind of network file system in operation before. I was proposing what I think is the best way to handle a new creation, created from the ground up.
Dave Yeo:
--- Quote from: Richard Crowley on December 13, 2017, 11:37:39 pm ---Thank you for responding. I do use JFS on big partitions but my "network" consists of a lot of different oddly-sized stuff from various. A few are 20 years old and the rest are at least ten years old. One of my most reliable drives has only 60GB on it so I really wondered. It was sold by apple and I usually use HFS+ on my Mac stuff.
My real struggle is whether use JFS rather than FAT32 (Windows) or HFS+ for this conglomeration of old and older disks. I referenced ZFS (from Oracle because I thought it would be an example of how best to what I want to do: speed, ,managing devices and preventing loss of data/data corruption. I was hoping that JFS had something comparable that would make my network faster and safer (ZFS being an example). I like a b-tree structure but also like the way FAT32.
--- End quote ---
You really don't want to use FAT32 if you can avoid it, though for sharing files between OSes, it is needed. I don't know much about HFS+. JFS is considered an excellent all around file system, at least according to Wikipedia and some years back I benchmarked JFS vs HPFS and in most cases JFS was faster. HPFS is limited to 2MBs of cache (HPFS386 can use much more) and as I said, has long chkdsk times.
Another consideration is that for OS/2 you really need EA (xttrs) support. Might work on HFS+, FAT32 has a kludge on OS/2 to support them but some report it leads to instability.
--- Quote ---Would it be better to simply run regular backups with something like a cron script?
--- End quote ---
Likely. Lots of people are using rsync for backups with good results, and of course there is zip.
Navigation
[0] Message Index
[#] Next page
Go to full version