• Welcome to OS2World OLD-STATIC-BACKUP Forum.
 

News:

This is an old OS2World backup forum for reference only. IT IS READ ONLY!!!

If you need help with OS/2 - eComStation visit http://www.os2world.com/forum

Main Menu

NLS, using ".msg" files in rexx?

Started by warpcafe, 2009.10.19, 17:55:22

Previous topic - Next topic

warpcafe

Hi all,

let me start like this: I hate to reinvent the wheel.
For a new rexx program I'm working on, I was aiming to have multiple language support.

What I actually wanted to do is put all string resources which are required by the program into a MSG file and simply have according different msg files for each supported language. The program then simply (well, more or less...) determines the language and selects the according file. Like in
- myapp_de.msg
- myapp_en.msg

Whatever. Even fully qualified "lang" settings a la en_us vs. en_gb or de_de / de_at could be possible, this is not the problem... the problem is:
How can I access such files from rexx?

Now, various gui-Rexx solutions like DrDialog or Vx-REXX have the ability to load "resources" from a DLL. However they only support icons or bitmaps. I am almost sure that DLLs can also include string resources, however even if this is the case, I am unable to find a way to access them by REXX.
So I went for the .msg files. Same here: I can't find an existing solution to access them by rexx.

(Note: Yeah, I know I could invent my own flavor of that... I already did that and it worked out nice. But I wanted to save my program from including unneeded bloated stuff by useless reinvention of wheels... and also I am not sure if "my" blend of msg file is able to cope with DBCS... didn't check that... Also it's a question of concentration to the necessary stuff: There's already enough to code to get the actual "work" done, so I would appreciate to save myself from doing that msg-by-hand-stuff at least.)

Anyone can help me out with that please?

Thanks for any hints,
Thomas
"It is not worth an intelligent man's time to be in the majority.
By definition, there are already enough people to do that"
- G.H. Hardy

RobertM

I read someplace that the support exists in REXXUTIL... but cant confirm that. Back around 1997 (before NLS support was added to REXXUTIL) someone wrote a REXX procedure to deal with them: http://www.rexxla.org/events/1997/report.html

If I can dig up more info on either, or some other method, I will letcha know.

Best,
Rob


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


herwigb

Alex Taylor has a very nice set of routines which do exactly that. They are used in EVFSGUI.

I asked Alex and got the permission to use his code. All the Samba GUI tools make use of this framework.

It can be found http://svn.netlabs.org/samba/browser/branches/guitools-1.0/shared/nlv.vrs
Kind regards,
HerwigB.

warpcafe

Hi Robert, Herwig,

many thanks for opening my eyes. In fact, I never noticed the existence of SysGetMessage! :) Stupid me.
On the other hand, it proves my assumption that something "must exist" already... ;)

Since we're at it... does anyone have an idea about the performance of SysGetMessage? Is it "fast" enough to load all captions, hints and texts for all controls of an entire dialog without slowing down the initial dialog display too much?

What I really appreciate about SysGetMessage is that it's able to handle parameter replacing out of the box. Wow, yet something else I don't need to reinvent :) and regarding Alex stuff I am more than pleased to see it being done in Vx-Rexx. That matches the dialect used by me, hehe... :))

The only "nasty" thing is that I would need to use message numbers. I was thinking of something smarter using a built-in lookup table for control/dialog names... so to say that I can tell it "load all strings for dialog 'xyz' " and it would return me a stem containing name, property and value like
"pb_1:caption:Save"
"pb_1:hint:Saves current record"
"dt_1:caption:This is a sample"
... which I would then simply fire into "vrset" statements and done. Well, I think I'll have to balance the benefits of SysGetMessage versus the efforts to manage message numbers. We'll see... :)

Again, thanks a ton to you guys,
Cheers,
Thomas
"It is not worth an intelligent man's time to be in the majority.
By definition, there are already enough people to do that"
- G.H. Hardy

herwigb

The message numbers are really the only drawback. I always have to remind myself to add a short comment as it makes appliying changes later MUCH easier. And I forget it way too often :'(
Kind regards,
HerwigB.

jep

Thomas... and all of you.

While SysGetMessage is the usual way to go, I'd suggest a Zip-file with multiple language files (plain text).
Similar to how LANGE and OpenOffice.org files work.

I found it out when I wrote eFTBE, that the built in rexx support inside unzip.dll support the neat feature to extract data to a stem that one can use as-is or process futher.

Wish that unrar.dll would contain something similar as well.

=====8<------------------
cfg.msg.try_odbc = 'Testing ODBC connection... One moment please!'
cfg.msg.connection_failed = 'Could not connect to the database'
cfg.msg.username_by_hand = 'Username: '
cfg.msg.password_by_hand = 'Password: '
cfg.msg.add_rec_tables = 'Adding recreated table'
cfg.msg.temp_odbc_driver_not_deleted = 'Tempory driver ('||ARG(2)||') still registered'
cfg.msg.add_tables = 'Adding tables'
cfg.msg.analysis = 'Analysing damaged table '||ARG(2)
cfg.msg.assertion = 'Database Assertion failure detected, restaring database...'

--------------->8========

/* Note additional arguments will be absorbed and used even though they're not visible, see text above with additional params ARG(2), ARG(3) etc. */
RxGetMsg: Procedure Expose cfg.
    interpret 'retval = '||value( 'cfg.msg.'||ARG(1) )
Return retval

ReadNLS: Procedure Expose cfg.
    tot_size = Stream( 'NLS.'||cfg.language, 'C', 'QUERY SIZE' )
    do while lines( 'NLS.'||cfg.language ) > 0
        parse value linein( 'NLS.'||cfg.language ) with pre' = 'post
        if pos( pre, post ) > 0 then do
            parse value post with .'||'post'||'next'||'last
            if length( next ) > 0 then
                interpret 'next = 'next
            else next = ''
            post = value( value( 'pre' ) )||d2c(13)||d2c(10)||value( 'post' )||next||value( 'last' )
        end
        interpret pre' = post'
    end
Return


One can also create text files and separate each row with a ";" (without the quotation marks), read and interpret it all in one go.
interpret charin( 'NLS.'||cfg.language, 1, tot_size )
But it may have to contain some more/other tweaks I don't remember now, as well.

//Jan-Erik

warpcafe

Hi jep,

that's a very interesting way to do it... but I am afraid that when I have all captions, text, messages and stuff in a file for an entire project with 3 or more dialogs and all their respective controls, it will become quite time-consuming to do all the interpretation at run-time.
While your approach would solve the issue of "how to link" from a control to its needed string resource in a file, it also exposes too much of the program logic to the end-user, which in effect opens the door for errors (or even "security risks") in my mind. Okay, the security risk is neglectable on my end perhaps, since it would break the prorgram and make it unusable for the user, so ther is no intention by a user to fiddle with the cfg file.

Nevertheless, the flexibility of the solution has some price to pay at run-time. I would prefer to have simple plain-text input files (also for the translators) and a very small, fast and straightforward "loading" at run-time. In between these phases, a highly complicated process can "compile" the text input to the desired format for runtime.

Regarding the unzip - that's indeed nice as well, however unzipping something into memory at each run-time doesn't make sense to me in this context, also it means trading-off the benefits of using SysGetMessage (this won't work anymore with unzipping into a stem).

Jep, don't get me wrong: What you showcased here is highly flexible and a very interesting approach indeed (I never thought of doing it that way). However, for my needs it puts a mis-balance to performance and processing requirements when comparing the main task of the program versus the "feature" of multilingual resource handling.

Basically, the SysGetMessage is almost perfect for my needs. The only drawback is that I would have to know the resource ids (message numbers) by heart when doing the coding of the application. All I want to achieve is to be able to handle this part a little smarter... but I'm looking into a solution for it.

Thanks a lot for the suggestions anyway - I am sure they will become much more important in different projects than the one I am working on now.

Cheers,
Thomas
"It is not worth an intelligent man's time to be in the majority.
By definition, there are already enough people to do that"
- G.H. Hardy

RobertM

Quote from: warpcafe on 2009.10.20, 16:38:00
Hi jep,

that's a very interesting way to do it... but I am afraid that when I have all captions, text, messages and stuff in a file for an entire project with 3 or more dialogs and all their respective controls, it will become quite time-consuming to do all the interpretation at run-time.

You may be surpised. I'm using REXX on an ancient Athlon 1.1GHz machine to parse 3 catalog files containing over 90,000 records.  The records are in pipe delimited format (plain text) a few thousand characters long per record. It takes about .003 seconds per record to parse. Or .3 seconds per 100 records.

The parsing stage includes finding various information in the data string for each record, (which each of is a few thousand characters long), create a new set of variables for each entry in the record string, create a new set of variables for each set of additional information I am interested in (it's laptop computer descriptions, and lacks specific fields for HDD size, screen size, etc; so the script has to find that information in the description string: "Comes with a 17.1" WXGA LCD screen and a 120GB HDD") - about 40 variables per record, create an SQL insert statement, write that statement to a separate file and repeat for all 90,000+ records.

(after that, I dump the whole file to MySQL to create the database without having to do individual records)

So, I've found, even with the disk reads (across the network from the file on the server to the workstation that is processing it) of the text files and the disk writes  (also across the network from the file on the server to the workstation that is processing it) of the SQL command file, that .003 seconds for each record is pretty darn fast.

You may also find that such speeds are sufficient for any file/text parsing you need. I am guessing that if you remove the write stage that you probably don't need, and have the files accessed locally, you would see even better results. That should hopefully offset the unzip time, especially as you can use the unzip dll to only load which files you need.

The bigger concern in either method is that stem access/creation/changing in ObjectREXX is pathetic when compared to the same exact code in cREXX. And introduces memory leaks and eventual "Out of resources" errors with only a small amount of memory used.

Quote from: warpcafe on 2009.10.20, 16:38:00
While your approach would solve the issue of "how to link" from a control to its needed string resource in a file, it also exposes too much of the program logic to the end-user, which in effect opens the door for errors (or even "security risks") in my mind.

What if the zip file was encrypted? I think the unzip.dll supports reading from encrypted files (but could be wrong).

Quote from: warpcafe on 2009.10.20, 16:38:00
Okay, the security risk is neglectable on my end perhaps, since it would break the prorgram and make it unusable for the user, so ther is no intention by a user to fiddle with the cfg file.

Nevertheless, the flexibility of the solution has some price to pay at run-time. I would prefer to have simple plain-text input files (also for the translators) and a very small, fast and straightforward "loading" at run-time. In between these phases, a highly complicated process can "compile" the text input to the desired format for runtime.

Regarding the unzip - that's indeed nice as well, however unzipping something into memory at each run-time doesn't make sense to me in this context, also it means trading-off the benefits of using SysGetMessage (this won't work anymore with unzipping into a stem).

Well, it would require a little additional work, but probably could be done.


Quote from: warpcafe on 2009.10.20, 16:38:00
Jep, don't get me wrong: What you showcased here is highly flexible and a very interesting approach indeed (I never thought of doing it that way). However, for my needs it puts a mis-balance to performance and processing requirements when comparing the main task of the program versus the "feature" of multilingual resource handling.

Basically, the SysGetMessage is almost perfect for my needs. The only drawback is that I would have to know the resource ids (message numbers) by heart when doing the coding of the application.

Nah... just create a "lookup table" and use a variable. When used in a stem, the "sub-variable" becomes the value. I think the same can be done very easily with regular (non-stem) variables.

DialogInfo.DialogOne.Caption for instance can be the name you use in your program, then simply "superclass" the call to SysGetMessage so that you send it the above string as text. It can interpret out the value, file and so on from the stem sections.

I use a similar method to dynamically built variables for a report for a REXX program I have written to calculate data from about 10,000 records. The stem creation and accessing takes fractions of seconds. The calculations on the other hand take a little longer. All in all, it creates the whole report, html and all, in about 8-15 seconds. I doubt you'd be using one hundredth of that number of records, making the time it takes for you to accomplish something probably 1/500th of that time (as I also doubt you will be doing a bunch of math calculations, building a 50,000 line html file in memory, etc).

Best,
Rob


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


warpcafe

Heya,

there's 2 things that come together here:

First, my resentments concerning "too much string parsing involved when low-level batch processing is wanted" comes from experiences I made when we had to deep-analyze execptions from java batch logs. At that time, I had the free choice of creating/using whatever I wanted to optimize the manual analytics... so the decision was clear that for these purpose I wanted to use my preferred "scripting language" rexx.
I created some prototypes and gave them a try... and everything looked cool. Later, it turned out that rexx showed some strange hiccups, memory leaks and obvious errors when reading and parsing back/forth throughout the file. Okay, these were files of about 1 to 1.5 GiB size... but again, I wouldn't have minded if the processing took ages. I was rather disappointed to see it not working as expected and showing different behaviour on huge files compared to small files. I could have tried using different rexx "dialects" or environemnts, even platforms... however that effort was out of scope, so I chunked the processing into a couple of aggregation steps et cetera... finally it worked. But a dull feeling of general mistrust lingers in the back of my head since then. ;)

Still I am impressed by the flexibility and speed of rexx string functions. Nothing on this planet compares to it. And I use them when I think it makes sense. However, sometimes I feel that using flexible and fast string parsing and stuff is nothing but a pretext to hide the fact that the developer has no idea of how to solve proecssing tasks in a "SMART" way :)

Side-note:
I remember back in the 80s and early 90s I created COBOL programs that served similar needs. We didn't had these all-in-wonder parsing thinggies at our hands then... however the programs were blazingly fast and did the job accurately.
And you know, when our company's developers nowadays sometimes come to show-off what "incredibly fast" batch processing they did on their SQL tables and get too excited about their achievements, I make a joke of telling them that I achieved a 100-times faster processing with a fraction of the code in the 80s. ;) Okay, it might be a bit unfair to compare the two things, but at the end of the day - it's all about processing and moving data, isn't it?

Secondly, interpreting things at run-time (thus using rexx's "built-in" capabilities) is certainly something cool and might prove to be the best approach in a situation... but... who on Earth can tell exactly what this thing will do if the data to be processed is not like expected? It will interpret something, OK. What does that mean? Usually I would expect a processing step to be of binary nature: OK or Error. Now, what will happen if the INTERPRET is sumbitted something that is not "like expected"... will this cause an error? Will simply "nothing" happen? There's too much guesswork involved in my mind... and let's face it: It can be very hard to understand, not to mention a lack of self-descriptiveness.

Just my 2ct.

BTW: I found what I believe to be a "smart" solution for my problem. You are correct when you suggest to use a lookup table kind of thing - that's exactly what I was talking about (so I see you understood what the problem is :) ).
My solution for the lookup table is... an INI file, like in registry. The benefit in my mind is that it has proven stability, acceptable speed, plus the benefits of flexibility and being supported "natively" by rexx / vx-rexx.
It will allow me to directly access one specific controls data as well as comfortably gathering everything contained under a "parent" control (e.g. data of all controls in a dialog). Then, I only need to return the message number for the message file and am able to use the standard SysGetMessage.
The "source" file will contain a simple decsription like control:property:value and this will then be transferred into two files - the MSG file with synthetical msg numbers... and the INI file that links these numbers to the controls... get the picture? I think this is the best way make use of "standards" (i.e. no additional wheel invention, no additional DLL) to achieve the desired felxibility and feature set.

Cheers,
Thomas
"It is not worth an intelligent man's time to be in the majority.
By definition, there are already enough people to do that"
- G.H. Hardy

RobertM

Nice solution - and yes, memory leak issues have plagued me (at the very least with OREXX). So, very nice method you've chosen.

Best,
R


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|