Author Topic: Start a program using utf-8 as system codepage  (Read 4239 times)

Remy

  • Hero Member
  • *****
  • Posts: 646
  • Karma: +9/-1
    • View Profile
Start a program using utf-8 as system codepage
« on: February 11, 2023, 02:24:51 am »
Hi,

Is there a was to start a program in an OS/2 env using utf-8 codepage ?
Under RXULS inf doc, os/2 supported codepage are listed but the corresponding utf-8 isn't supported as default or alternate os/2 env codepage.

Waiting suggestions.

Traget: Read utf-8 chars like japanese or other specific characters not having a readable code under default 850 codepage from audio metadatas.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4805
  • Karma: +99/-1
    • View Profile
Re: Start a program using utf-8 as system codepage
« Reply #1 on: February 11, 2023, 02:54:08 am »
Use an editor that can display unicode such as Alex's qe.exe or featherpad. Or a browser.

Alex Taylor

  • Sr. Member
  • ****
  • Posts: 389
  • Karma: +5/-0
    • View Profile
Re: Start a program using utf-8 as system codepage
« Reply #2 on: February 11, 2023, 04:13:08 am »
OS/2 doesn't support UTF-8 as a process codepage. 

It does support it as a PM message-queue codepage, but this has to be set after the program starts up and creates a message queue.  You can force a change to a running PM app's message-queue codepage using CPPal (https://hobbes.nmsu.edu/?detail=%2Fpub%2Fos2%2Futil%2Fwps%2FCpPal_0-30.zip) - in most cases this requires you to also reset the window font afterwards (e.g. using the Font Palette) to force a refresh.

However, this technique has limited usefulness, mainly because the standard PM controls aren't really designed to handle UTF-8. Static text controls may be semi-legible, but text fields will probably just show an unreadable overlapping mess.

UTF-8 (1208) is supported as a GPI direct-rendering codepage, but it's apparently poorly optimized because rendering speed is atrociously slow. Raw UCS-2 (1200) is much faster to render in GPI, so in most cases it's better to internally convert UTF-8 to UCS-2 and render it that way. I once started to write a MLE control that could support Unicode encoded text (the source, such as it is, is still somewhere on my Github site), but I eventually found it was easier to just use Qt.

In most cases, if you want to render UTF-8 text (or any Unicode encoded text, actually) it's almost always preferable to use an alternative rendering library based on FreeType, like Cairo or Pango, as Qt and the Mozilla apps do. 

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 4745
  • Karma: +41/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Start a program using utf-8 as system codepage
« Reply #3 on: February 11, 2023, 03:00:38 pm »
FYI: On the latest Warpstock Alex taught me how to use CpPal and I documented here to don't forget it: "Displaying an Application with Japanese Characters Correctly"

Regards
« Last Edit: February 11, 2023, 03:04:27 pm by Martin Iturbide »
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Remy

  • Hero Member
  • *****
  • Posts: 646
  • Karma: +9/-1
    • View Profile
Re: Start a program using utf-8 as system codepage
« Reply #4 on: February 11, 2023, 03:23:49 pm »
Use an editor that can display unicode such as Alex's qe.exe or featherpad. Or a browser.

How will PM123 display these audio metadatas ?

Alex Taylor

  • Sr. Member
  • ****
  • Posts: 389
  • Karma: +5/-0
    • View Profile
Re: Start a program using utf-8 as system codepage
« Reply #5 on: February 11, 2023, 07:48:57 pm »
FYI: On the latest Warpstock Alex taught me how to use CpPal and I documented here to don't forget it: "Displaying an Application with Japanese Characters Correctly"

Nice article, but it could benefit from some tweaks. If you've modified CONFIG.SYS to have 932 as secondary codepage, you really want to change the process codepage (which will work in this case).  So in CPPal you'd want to select "secondary codepage" before dragging. And, in fact, if you've got 932 as a secondary codepage you technically don't need CPPal at all, you can just run the program from a session which does 'chcp 932' before starting the program.  (The program object's Language tab also lets you do this.)

CPPal is really useful when you don't have the secondary codepage set, but in that case the results aren't great if you're changing from a single-byte to a double-byte codepage.

Alex Taylor

  • Sr. Member
  • ****
  • Posts: 389
  • Karma: +5/-0
    • View Profile
Re: Start a program using utf-8 as system codepage
« Reply #6 on: February 11, 2023, 07:52:30 pm »
Use an editor that can display unicode such as Alex's qe.exe or featherpad. Or a browser.

How will PM123 display these audio metadatas ?

The CPPal technique might work here, but I doubt the results would be great. Ideally, PM123 should support it internally, I'm not sure if it has that logic or not (I don't generally use it myself). If not, it probably wouldn't be that difficult for a programmer to add.