Author Topic: HPFS and JFS internal definition specifications  (Read 2655 times)

John Poltorak

  • Sr. Member
  • ****
  • Thank You
  • -Given: 20
  • -Receive: 1
  • Posts: 335
    • View Profile
Re: HPFS and JFS internal definition specifications
« Reply #30 on: June 18, 2017, 03:05:51 pm »
I want to take a look at JFS and ZFS as well.

Thank you,
Rick C. Hodgin

ZFS is gaining interested in FreeBSD land so I guess the source for it must be available...

Eric

  • Newbie
  • *
  • Thank You
  • -Given: 1
  • -Receive: 0
  • Posts: 8
    • View Profile
Re: HPFS and JFS internal definition specifications
« Reply #31 on: June 19, 2017, 04:19:09 am »

ZFS is gaining interested in FreeBSD land so I guess the source for it must be available...

Not just FreeBSD.. It's somewhat popular on Linux too (http://zfsonlinux.org/).  On both it's pretty darn stable.  Due to its license, though, it cannot be put into the Linux kernel.

I think Linux and FreeBSD have both gone down different paths and have slightly different ZFS features (although I think the pools + filesystems can be set to a level that maintains compatibility).  Sources for both are available though.

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 14
  • -Receive: 4
  • Posts: 159
    • View Profile
    • Liberty Software Foundation
Re: HPFS and JFS internal definition specifications
« Reply #32 on: June 21, 2017, 12:30:19 am »
I've been thinking about it.  I can no reason whatsoever to not work with a journaling file system in moving forward.  It is the only path that makes sense for a long-term system.

I'll be looking at JFS in moving forward.

Thank you,
Rick C. Hodgin

John Poltorak

  • Sr. Member
  • ****
  • Thank You
  • -Given: 20
  • -Receive: 1
  • Posts: 335
    • View Profile
Re: HPFS and JFS internal definition specifications
« Reply #33 on: June 29, 2017, 10:08:51 am »

ZFS is gaining interested in FreeBSD land so I guess the source for it must be available...

Not just FreeBSD.. It's somewhat popular on Linux too (http://zfsonlinux.org/).  On both it's pretty darn stable.  Due to its license, though, it cannot be put into the Linux kernel.

I think Linux and FreeBSD have both gone down different paths and have slightly different ZFS features (although I think the pools + filesystems can be set to a level that maintains compatibility).  Sources for both are available though.

For info on ZFS see:- open-zfs.org

FreeBSD has incorporated ZFS for 10 years -  https://wiki.freebsd.org/ZFS

ZFS on Linux seems like a real  mess.... Looking at http://zfsonlinux.org/ there have been 11 changes since the previous release - so that doesn't suggest a very stable product... and then you have to choose which flavour of Linux you want to use: Arch, Debian, Fedora, Gentoo, openSUSE, RHEL & Centos, Ubuntu... Can of worms if you ask me....

John Poltorak

  • Sr. Member
  • ****
  • Thank You
  • -Given: 20
  • -Receive: 1
  • Posts: 335
    • View Profile
Re: HPFS and JFS internal definition specifications
« Reply #34 on: June 29, 2017, 11:26:32 am »

ZFS is gaining interested in FreeBSD land so I guess the source for it must be available...

Not just FreeBSD.. It's somewhat popular on Linux too (http://zfsonlinux.org/).  On both it's pretty darn stable.  Due to its license, though, it cannot be put into the Linux kernel.

I think Linux and FreeBSD have both gone down different paths and have slightly different ZFS features (although I think the pools + filesystems can be set to a level that maintains compatibility).  Sources for both are available though.

For info on ZFS see:- open-zfs.org

FreeBSD has incorporated ZFS for 10 years -  https://wiki.freebsd.org/ZFS

ZFS on Linux seems like a real  mess.... Looking at http://zfsonlinux.org/ there have been 11 changes since the previous release - so that doesn't suggest a very stable product... and then you have to choose which flavour of Linux you want to use: Arch, Debian, Fedora, Gentoo, openSUSE, RHEL & Centos, Ubuntu... Can of worms if you ask me....

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 14
  • -Receive: 4
  • Posts: 159
    • View Profile
    • Liberty Software Foundation
Re: HPFS and JFS internal definition specifications
« Reply #35 on: July 11, 2017, 05:50:37 pm »
Does anybody know how HPFS or JFS handles a file that's been moved from one directory to another?

As I understand it, the goal of HPFS is to keep files close to their root nodes so that far away disk access is minimized.  If a file existed in one location, and needed to be moved to another, would it physically be migrated on disk?  Would there be a size or space policy (so that if it's small enough, or there's enough disk space it's moved, but if it's too big or there isn't enough space it's not moved)?

Thank you,
Rick C. Hodgin

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 61
  • -Receive: 10
  • Posts: 322
    • View Profile
Re: HPFS and JFS internal definition specifications
« Reply #36 on: July 11, 2017, 07:18:41 pm »
Does anybody know how HPFS or JFS handles a file that's been moved from one directory to another?

If a file existed in one location, and needed to be moved to another, would it physically be migrated on disk?  Would there be a size or space policy (so that if it's small enough, or there's enough disk space it's moved, but if it's too big or there isn't enough space it's not moved)?

DIY. MOVE a large file to another directory, disk almost full or not. Large enough to notice the difference between an administrative solution and actually copying a few MiBs. Next try to MOVE a file to another drive.

I'm expecting an efficient administrative solution, and finally MOVE not being able to copy and delete a source file. Yet another clue is that MOVE won't have a /Verify option, like the /V of COPY.

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 14
  • -Receive: 4
  • Posts: 159
    • View Profile
    • Liberty Software Foundation
Re: HPFS and JFS internal definition specifications
« Reply #37 on: July 11, 2017, 07:25:47 pm »
Does anybody know how HPFS or JFS handles a file that's been moved from one directory to another?

If a file existed in one location, and needed to be moved to another, would it physically be migrated on disk?  Would there be a size or space policy (so that if it's small enough, or there's enough disk space it's moved, but if it's too big or there isn't enough space it's not moved)?

DIY. MOVE a large file to another directory, disk almost full or not. Large enough to notice the difference between an administrative solution and actually copying a few MiBs. Next try to MOVE a file to another drive.

My question is more about if anyone knows what the design policy is for HPFS or JFS, because I can see to do it multiple ways and some of them would be more efficient in the long-term, though less-efficient in the short-term.

I have dyslexia and am not good at reading through pages and pages of documentation to find answers.  I try and ask and use other people's knowledge before I dig deep, as they can often say, "There's a chapter called ... you can read" and then I hone in on my target.

I think I'll add a command-line option which force-moves a file to be near to the other directory node, and use the non-move method by default, and the force-move method when asked to do so.

Thank you,
Rick C. Hodgin

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 61
  • -Receive: 10
  • Posts: 322
    • View Profile
Re: HPFS and JFS internal definition specifications
« Reply #38 on: July 12, 2017, 12:34:50 am »
My question is more about if anyone knows what the design policy is for HPFS or JFS, because I can see to do it multiple ways and some of them would be more efficient in the long-term, though less-efficient in the short-term.

DosMove is an OS facility:

Quote
DosMove can be used to change only the name of a file or subdirectory, allowing the file object to remain in the same subdirectory. Global file-name characters are not allowed in the source or target name.

If the specified paths are different, the subdirectory location of the file object is changed also.

Of course an IFS can do anything to mimic the expected behaviour. There's no requirement to not touch the file itself. But, as stated earlier, with a DosCopy one of the missing options is the potion to verify the copied file.

I think I'll add a command-line option which force-moves a file to be near to the other directory node, and use the non-move method by default, and the force-move method when asked to do so.

Sometimes I do copy a file to a different drive, delete the original file, copy it back, and delete the copy. That's the only structure-related strategy I'm using, but not that frequently.

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 14
  • -Receive: 4
  • Posts: 159
    • View Profile
    • Liberty Software Foundation
Re: HPFS and JFS internal definition specifications
« Reply #39 on: July 12, 2017, 01:18:22 am »
Sometimes I do copy a file to a different drive, delete the original file, copy it back, and delete the copy. That's the only structure-related strategy I'm using, but not that frequently.

That's definitely the safest way to do it.  I really think in most cases it won't matter much.  I was just sitting there pondering how to do various things, getting a mental image in my head of the overall procedures, and that one occurred to me.  I think I have a pretty good handle on everything else.

Thank you,
Rick C. Hodgin

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 8
  • -Receive: 98
  • Posts: 1168
    • View Profile
Re: HPFS and JFS internal definition specifications
« Reply #40 on: July 12, 2017, 01:56:43 am »
As far as I know, when simply moving a file to a different directory on the same partition, just the directory structure is updated, resulting in the move being close to instant. I don't know about JFS, but I believe HPFS will adjust its directory tree during idle time to be a balanced B-tree.
The (early) JFS source is available, though being GPL, perhaps depending on what license you're thinking of using, it might be better not to look.

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 14
  • -Receive: 4
  • Posts: 159
    • View Profile
    • Liberty Software Foundation
Re: HPFS and JFS internal definition specifications
« Reply #41 on: July 12, 2017, 01:58:57 am »
As far as I know, when simply moving a file to a different directory on the same partition, just the directory structure is updated, resulting in the move being close to instant. I don't know about JFS, but I believe HPFS will adjust its directory tree during idle time to be a balanced B-tree.
The (early) JFS source is available, though being GPL, perhaps depending on what license you're thinking of using, it might be better not to look.

Yeah.  I'm avoiding any existing code unless it's in the public domain.  My goal is to create a ground-up driver that is truly in the public domain.

Thank you,
Rick C. Hodgin

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 61
  • -Receive: 10
  • Posts: 322
    • View Profile
Re: HPFS and JFS internal definition specifications
« Reply #42 on: July 12, 2017, 02:37:04 am »
just the directory structure is updated, resulting in the move being close to instant.

That's DosMove(). Any alternative which uses DosCopy(), or equivalents to copy a file, instead of DosMove() can and will result in unexpected errors, like a SYS112. So:

Code: [Select]
COPY + DEL != MOVE

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 61
  • -Receive: 10
  • Posts: 322
    • View Profile
Re: HPFS and JFS internal definition specifications
« Reply #43 on: August 05, 2017, 02:49:17 pm »
I think I'll add a command-line option which force-moves a file to be near to the other directory node, and use the non-move method by default, and the force-move method when asked to do so.

Sometimes I do copy a file to a different drive, delete the original file, copy it back, and delete the copy. That's the only structure-related strategy I'm using, but not that frequently.

FWIW: you can use a simple force-move method by default when two drives are involved. Now moving a file to a different drive, by using MOVE, is a syntax error.

Apparently the API call DosError(0) isn't used, because the drive has to be ready before the syntax error is triggered. So "MOVE C:CONFIG.SYS B:" wants you to insert a diskette before MOVE generates the final SYS1003 error. That can be improved too. If there's a syntax error, then making the user insert a diskette is useless.

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 14
  • -Receive: 4
  • Posts: 159
    • View Profile
    • Liberty Software Foundation
Re: HPFS and JFS internal definition specifications
« Reply #44 on: August 05, 2017, 03:06:24 pm »
I think I'll add a command-line option which force-moves a file to be near to the other directory node, and use the non-move method by default, and the force-move method when asked to do so.

Sometimes I do copy a file to a different drive, delete the original file, copy it back, and delete the copy. That's the only structure-related strategy I'm using, but not that frequently.

FWIW: you can use a simple force-move method by default when two drives are involved. Now moving a file to a different drive, by using MOVE, is a syntax error.

Apparently the API call DosError(0) isn't used, because the drive has to be ready before the syntax error is triggered. So "MOVE C:CONFIG.SYS B:" wants you to insert a diskette before MOVE generates the final SYS1003 error. That can be improved too. If there's a syntax error, then making the user insert a diskette is useless.

I have goals to complete a full ground-up rewrite of the OS/2 kernel.  I won't be able to do it unless I'm able to figure out a way to devote my daytime labor hours to it.  I'll be 48 years old this month, and I just can't work after normal business hours like I've been able to in the past.

I'm trying to get a business going where I can manufacture and sell old IBM Model-F style keyboards, but with more keys.  If that business is successful, or if I'm able to find a different successful business model, then I'll be able to use those daytime labor hours for this project, and then this project stands a chance at succeeding.  Another option would be to have others come on board and help me with the ground-up effort.  I would welcome such help.

This pursuit of my own ground-up operating system has been my life's greatest goal.

Thank you,
Rick C. Hodgin
« Last Edit: August 05, 2017, 03:39:38 pm by Rick C. Hodgin »