Author Topic: Can we store UTF-8 encoded file names on JFS? (turns to git and ssh problems)  (Read 1495 times)

Andi B.

  • Hero Member
  • *****
  • Posts: 916
  • Karma: +16/-2
    • View Profile
I want to synchronize documents with a Linux system via git. I've some file names with German 'Umlaute' (äüöß...). When I check out the repository on Linux there I get something like 'invalid encoding' with the file managers there. Of course I can convert the file names to UTF-8 (convmv -f cp850 -t utf-8 *) but I fear I can't handle these files afterward on OS/2.

So how do deal with that? Other solutions then changing file names to not use 'special' characters outside ANSI code page?

(Edit: change subject)
« Last Edit: May 27, 2025, 12:07:31 pm by Andi B. »

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5417
  • Karma: +129/-1
    • View Profile
Re: Can we store UTF-8 encoded file names on JFS?
« Reply #1 on: May 25, 2025, 08:39:05 pm »
JFS uses UTF8 for encoding, though I guess it gets converted to/from your codepage when read/write.
I think the problem is actually git, which seems to encode in the native codepage. I committed an authors name that used umlauts, seemed to work fine with OS/2, Martin looked at the repository on I assume Windows and the name looked wrong, he changed it and then it was broken on OS/2 but looked correct in web browser using UTF8.
Searching, it seems you can use .gitattributes to control encoding, eg https://stackoverflow.com/questions/48907049/what-is-the-appropriate-character-encoding-for-a-git-repo or the official documentation, https://git-scm.com/docs/gitattributes/2.18.0
I didn't know about this and need to remember it for similar reasons. I do routinely use .gitattributes to make Rexx scripts have the correct line endings.
.gitattributes from the screensaver,
Code: [Select]
# CRLF is crucial for OS/2 REXX files (also fine for Windows .cmd)
*.cmd eol=crlf
*.txt eol=crlf
# Be safe by making the Warpin script (*.wis) have crlf
*.wis eol=crlf

Edit: you likely need to experiment, let us know the results
« Last Edit: May 25, 2025, 08:42:54 pm by Dave Yeo »

KO Myung-Hun

  • Jr. Member
  • **
  • Posts: 81
  • Karma: +11/-0
    • View Profile
Re: Can we store UTF-8 encoded file names on JFS?
« Reply #2 on: May 26, 2025, 05:45:57 am »
Hi/2.

JFS itself supports unicode although I'm not sure its UCS-2 or UTF-8. However, no OS/2 IO APIs do not support Unicode at all. Converting layer is necessary.

Anyway if you have problems only for file names, then try https://github.com/komh/git-os2/releases/tag/2.42.0-os2-r1.

Andi B.

  • Hero Member
  • *****
  • Posts: 916
  • Karma: +16/-2
    • View Profile
Re: Can we store UTF-8 encoded file names on JFS?
« Reply #3 on: May 26, 2025, 08:58:34 am »
Thanks.

Dave, for line endings I don't wanna convert I usually set in .git/config autocrlf=false. There is also =input or others but I did not experiment a lot with that. Maybe your solution with .gitattributes is better. Have to think about.

git - now I remember I had a very hard time with the git version on this system. I could not connect to my ssh repositories if I remember correctly. I had to downgrade to 2.0.x as the latest from netlabs gave me problems I cant overcome. Now I remember I wanted to solve this 'git version problem' with the new system I wanted to set up. Unfortunately I've strange problems with ArcaOS even during installation on that new system. So I have to do more and more work on Linux boxes instead new ArcaOS unfortunately.

KO Myung-Hun, what is fixed in your version instead the netlabs ones? Is it only the newer code compiled to OS/2 or do you remember special OS/2 fixes compared to other versions?

Heikki Kekki

  • Newbie
  • *
  • Posts: 28
  • Karma: +1/-0
    • View Profile
Re: Can we store UTF-8 encoded file names on JFS?
« Reply #4 on: May 26, 2025, 09:19:35 am »
Here I can store UTF-8 encoded file names on JFS and on ext4. Both appears okay on OS/2 (ArcaOS) including äüöß.srt. On Android TV äöå works fine from OS/2 and Xubuntu. 

KO Myung-Hun

  • Jr. Member
  • **
  • Posts: 81
  • Karma: +11/-0
    • View Profile
Re: Can we store UTF-8 encoded file names on JFS?
« Reply #5 on: May 26, 2025, 11:38:43 am »
It has
 
Quote
* Fixed UTF-8 encoded filename could not be created

See for details:
  https://github.com/komh/git-os2/releases


Andi B.

  • Hero Member
  • *****
  • Posts: 916
  • Karma: +16/-2
    • View Profile
Currently can't say anything about codepage (UTF-8) but the new git version from KO Myung-Hun fixes another 'git clone https...' issue I saw a few days ago.

But now I stumbled over a ssh problem which I think causing this git fail on my local (NAS) repositories. ssh -v ... now shows that it can not find the home directories. It shows something like 'can not find /E$/home/...' where it should look at E:/home/....

I think I saw this before and that's the reason why I settled on git2.0.0 and older ssh on another system. But newest from netlabs.exp (and .rel) do not work here on a fresh ArcaOS5.1DE system.

One thing I've already repaired is /noexistant/home. For this we have to set a home directory for user root in klusmgr. Samba user manager does not work either on this fresh ArcaOS5.1DE so had to use klusmgr for that :-(. You also had to set a shell for root there. I set /usr/bin/sh. Think this is good but don't know. Any thoughts