Author Topic: JFS file corruption  (Read 1444 times)

Greg Pringle

  • Full Member
  • ***
  • Thank You
  • -Given: 6
  • -Receive: 2
  • Posts: 118
    • View Profile
JFS file corruption
« on: October 27, 2019, 07:18:55 pm »
I use JFS for large volumes and need back ups. When doing this in the past I used XCOPY /S /E which worked well. Then when the disks became too big I use cp -r which worked with larger data sets. Then both started failing.
Then I wrote some Java to do the same thing.
First it creates a text file listing of every file and directory with full paths.
On a 2T drive this could be about a 150MB file.
Then I read the listing and create smaller files limited by the number of files and the data size.
This created 110 smaller listings.
These listings could be used to copy the files to any drive or subdirectory.
However when I went to copy the files I had a JFS sytem crash about 8 listings in.
After some review I determined the file being copied  was currupt on the disk but had passed chkdsk.
It turned out I had to remove the file from the listing and about 20 more after it.
This was done by ediiting the listing file and then restaring the copy.
This approach copied the files but not the extended attributes.

Then I took the copy on the new SSD drive and put it in a fast machine which had an SSD boot drive.
The copy failed again. The speed was too fast between the SSD volumes for JFS to sync up.
It turned out to be two reasons. First, there needed to be some throttling when writing 400MB files and second there was a java problem. The java RandomAccessFile does not actually write in real time like standard ioctl disk access. Most people rely on a simple close() to end the file write. This failes on very speedy systems. Instead a RandomAccessFile.getFD().sync() is required before the close().

To make my life harder the SSD had a fault at about half way where a write would cause a kernal panic. If I wrote past this part of the disk it worked fine. The disk was a brand new SanDisk 3D 1T drive.


ivan

  • Hero Member
  • *****
  • Thank You
  • -Given: 2
  • -Receive: 35
  • Posts: 1026
    • View Profile
Re: JFS file corruption
« Reply #1 on: October 27, 2019, 08:05:22 pm »
Hi Greg,

There you have found the reason why I never use a JFS partition larger than 500GB.  Having been bitten in the past by a corrupt 1 TB JFS partition I soon learned to keep the partitions down to a manageable size for easy backups.  A case of once bitten twice shy. 

The only place where I don't have multiple partitions is in my NAS boxes, those have RAID 1 disks formatted as either Ext3 or Ext4.

Greg Pringle

  • Full Member
  • ***
  • Thank You
  • -Given: 6
  • -Receive: 2
  • Posts: 118
    • View Profile
Re: JFS file corruption
« Reply #2 on: October 28, 2019, 01:36:39 pm »
Hi Ivan, that is good advice. Normally I do try and keep the partitions smaller. The problems I encountered were not a complete JFS failue but only a section of the disk. It looks like the file pointers became crossed for a section of files after a power outage.
In the past I have had 3 complete JFS corruption failures. Each one came from filling the JFS partition more than 90% and then continurally writing and deleting files with the near full partition. That will eventually kill the partition.
I have not tried the drive reorganization feature as a solution. Instead I don't do the > 90% activity or just copy the entire drive to a new one.
As to raid. I have used (now obsolete) the EzRaid devices which are raid 1 mirrors. These were nice devices because the disks were not changed by the controller and could be removed and used without the controller. Lately I have some StarTech PCI Express 2.0 SATA III 6Gbps RAID (rail 1) controllers on some drives. These mark the disks and make them only work with a similar controller. These use the Marvell - 88SE9230 controller which is also used by other manufacturers. The StarTech can be used for the boot drive. It has a built in setup utility accessed before the boot.
« Last Edit: October 28, 2019, 01:38:14 pm by Greg Pringle »

Paul Smedley

  • Sr. Member
  • ****
  • Thank You
  • -Given: 12
  • -Receive: 54
  • Posts: 474
    • View Profile
Re: JFS file corruption
« Reply #3 on: October 28, 2019, 10:01:21 pm »
Hi Greg,

I use JFS for large volumes and need back ups. When doing this in the past I used XCOPY /S /E which worked well. Then when the disks became too big I use cp -r which worked with larger data sets. Then both started failing.
Then I wrote some Java to do the same thing.

Have you considering using rsync to do this? rsync will only copy across the files that have changed - significantly reducing the duration/CPU impact of the copy.

My daily rsync of ~600GB of data on my NAS to an external drive runs in a couple of minutes.

Cheers,

Paul.

roberto

  • Sr. Member
  • ****
  • Thank You
  • -Given: 16
  • -Receive: 10
  • Posts: 472
    • View Profile
Re: JFS file corruption
« Reply #4 on: October 29, 2019, 10:08:13 pm »
Maybe this could be the problem you had. With copy and with xcopy this is currently happening.
In fact, if someone can tell me how I can execute the copy command, I appreciate it.
I enclose a photo with the copy command in green, and the result later in pink.

I saw this same problem the other day when running on a windows connected to a test samba server with arcaos, when creating a new folder from windows I gave it two different names on the samba server.


Greg Pringle

  • Full Member
  • ***
  • Thank You
  • -Given: 6
  • -Receive: 2
  • Posts: 118
    • View Profile
Re: JFS file corruption
« Reply #5 on: October 30, 2019, 12:51:46 pm »
Maybe this could be the problem you had. With copy and with xcopy this is currently happening.

The problem here is not with copy or xcopy. It appears to be with the desktop.
First rename the second version were both match to a new name.
Then you need to change the desktop name of the confused one to match the "Real Name".
After that you can change the names again using the normal methods
« Last Edit: October 30, 2019, 12:54:57 pm by Greg Pringle »

Greg Pringle

  • Full Member
  • ***
  • Thank You
  • -Given: 6
  • -Receive: 2
  • Posts: 118
    • View Profile
Re: JFS file corruption
« Reply #6 on: October 30, 2019, 01:00:42 pm »

My daily rsync of ~600GB of data on my NAS to an external drive runs in a couple of minutes.


This is definitely something to try.
In addition I have the need to make archival disk copies of complete disks which can be stored or moved to another machine.
When there is corruption of a range of files I have used my copy software to bypass the corruption. Not sure what rsync would have done.