Author Topic: Rexx -> IBM C  (Read 1866 times)

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 40
  • -Receive: 9
  • Posts: 432
    • View Profile
Rexx -> IBM C
« on: October 10, 2018, 11:36:50 am »
I've got a list of names, with about 10 possibly occuring characters which have to be converted to Netscape's HTML. I'm checking all characters and insert missing characters ("&": "amp", "á": "á"). The number of characters is known and limited.

The virtual code below works. Is this a normal way to do this? Should I be using "case" instead of a nested "if"? Several apps use the same code (and names), so should this become a DLL?

Execution speed is not a problem. Speed gains are nice-to-have, in this case.

Just checking, I don't want to get used to bad programming habits like using a sprintf(buf,"%s",text) instead of a strcpy(buf,text). I'm aware of the lack of comments, and so on. A global variable (buffer) is used to avoid arguments, TBH. With Rexx I'd use functions like INSERT and/or CHANGESTR, but I assumed that with C you'll have to insert characters the harder way.


Code: [Select]
void HTMLName(void)
{
int i,j,len;

len=strlen(buffer);

for (i=0;i<len;i++)
   {
   if (buffer[i]=='&')
      {             
      for (j=len+3;j-4>i;j--)
         buffer[j]=buffer[j-4];
      buffer[++i]='a';
      buffer[++i]='m';
      buffer[++i]='p';
      buffer[++i]=';';
      buffer[len+4]=0;
      len=strlen(buffer);
      }
   else if (buffer[i]=='á')
      {             
      for (j=len+6;j-7>i;j--)
         buffer[j]=buffer[j-7];
      buffer[i]='&';
      buffer[++i]='a';
      buffer[++i]='a';
      buffer[++i]='c';
      buffer[++i]='u';
      buffer[++i]='t';
      buffer[++i]='e';
      buffer[++i]=';';
      buffer[len+7]=0;
      len=strlen(buffer);
      }
   else if (buffer[i]=='¥')
      {             

      ...

      }
   else if (buffer[i]== ... )
      {             

      ...

      }
   }
return;
}

RickCHodgin

  • Guest
Re: Rexx -> IBM C
« Reply #1 on: October 10, 2018, 02:21:44 pm »
I've got a list of names, with about 10 possibly occuring characters which have to be converted to Netscape's HTML. I'm checking all characters and insert missing characters ("&": "amp", "á": "&aacute;"). The number of characters is known and limited.

The virtual code below works. Is this a normal way to do this? Should I be using "case" instead of a nested "if"? Several apps use the same code (and names), so should this become a DLL?

Execution speed is not a problem. Speed gains are nice-to-have, in this case.

Just checking, I don't want to get used to bad programming habits like using a sprintf(buf,"%s",text) instead of a strcpy(buf,text). I'm aware of the lack of comments, and so on. A global variable (buffer) is used to avoid arguments, TBH. With Rexx I'd use functions like INSERT and/or CHANGESTR, but I assumed that with C you'll have to insert characters the harder way.


Code: [Select]
void HTMLName(void)
{
int i,j,len;

len=strlen(buffer);

for (i=0;i<len;i++)
   {
   if (buffer[i]=='&')
      {             
      for (j=len+3;j-4>i;j--)
         buffer[j]=buffer[j-4];
      buffer[++i]='a';
      buffer[++i]='m';
      buffer[++i]='p';
      buffer[++i]=';';
      buffer[len+4]=0;
      len=strlen(buffer);
      }
   else if (buffer[i]=='á')
      {             
      for (j=len+6;j-7>i;j--)
         buffer[j]=buffer[j-7];
      buffer[i]='&';
      buffer[++i]='a';
      buffer[++i]='a';
      buffer[++i]='c';
      buffer[++i]='u';
      buffer[++i]='t';
      buffer[++i]='e';
      buffer[++i]=';';
      buffer[len+7]=0;
      len=strlen(buffer);
      }
   else if (buffer[i]=='¥')
      {             

      ...

      }
   else if (buffer[i]== ... )
      {             

      ...

      }
   }
return;
}

I don't think the code you have will work like you want it to.  You're scanning through buffer[ i ], and you're updating into buffer[ ++i ], meaning when you encounter "&" you'll be replacing it with "&amp;" and overwriting whatever would've come after.  So to address this you're moving everything over backward from the end, but how big was your allocation buffer to begin with?  You're likely to destroy memory control blocks doing this unless your allocated buffer just happened to have enough space to allow this expansion.

What you should do instead is allocate a bufout block, and rename your buffer to bufin, and then copy everything from bufin to bufout and when you encounter special characters, use the translation and inject the multiple characters.  You can also do this in a lazy fashion, meaning you don't need to allocate the output buffer until you do a first pass through to find out how big your output buffer needs to be.  If you don't find any special characters, no translation is required.  If so, you know exactly how many to add.

If you know your input buffer is ASCII-0 terminated, then you can change your loop, which will prevent the initial scan all the way through buffer to get the count.

Code: [Select]
for (i = 0; bufin[i] != 0; ++i)
{
    // Code here
}

You can also use switch if you are looking for single-character cues.  Make sure you add the breaks.  And you can use a default clause to just copy:

Code: [Select]
switch (bufin[i])
{
    case '&':
        // Code here
        break;

    case 'á':
        // Code here
        break;

    case '¥':
        // Code here
        break;

    ...

    default:
        bufout[j++] = bufin[i];
        break;
}

When you're finished, bufout will be the HTMLName version, and bufin can be discarded.

General form:
Code: [Select]
for (pass = 1; pass <= 2; ++pass)
{
    if (pass == 1)
    {
        // Counting how big bufout will need to be
        for (i = 0, newsize = 0; bufin[i] != 0; ++i)
        {
            switch (bufin[i])
            {
                case '&':
                    newsize += 5;   // size of "&amp;"
                    break;
                case 'á':
                    newsize += 8;   // size of "&accute;"
                    break;
                ...
                default:
                    ++newsize;
                    break;
            }
        }
        continue;

    } else if (newsize == i - 1) {
        // No expansion was required;
        bufout = NULL;

    } else if ((bufout = malloc(newsize + 1))) {
        // Populate our output buffer
        bufout[newsize] = 0;
        for (i = 0, newsize = 0; bufin[i] != 0; ++i)
        {
            switch (bufin[i])
            {
                case '&':
                    newsize += strcpy(bufout + newsize, "&amp;");
                    break;
                case 'á':
                    newsize += strcpy(bufout + newsize, "&accute;");
                    break;
                ...
                default:
                    bufout[newsize++] = bufin[i];
                    break;
            }
        }
    }

    // Free our input buffer
    if (bufout)    free(bufin);
    else           bufout = bufin;

}

Untested, but something like that.
« Last Edit: October 10, 2018, 06:24:44 pm by Rick C. Hodgin »

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 40
  • -Receive: 9
  • Posts: 432
    • View Profile
Re: Rexx -> IBM C
« Reply #2 on: October 12, 2018, 01:27:58 pm »
I don't think the code you have will work like you want it to.  You're scanning through buffer[ i ], and you're updating into buffer[ ++i ], meaning when you encounter "&" you'll be replacing it with "&amp;" and overwriting whatever would've come after.

Thanks, I'll apply the suggestions, if anything to avoid a weird coding style. Main ones: compile a new string to not save a few bytes or to mimic a Rexx function, and switch.

The &amp; code works (which should insert "amp;"), but the other code may be broken:

"Boussard&Gavaudan £" -> "Boussard&amp;Gavaudan £". I was expecting a &pound; as a final character now, but I'll have to check that specific character later. Not important; this is about not adopting a weird coding style due to mainly ported code. Fixing a bug, if any, should be easy. The on-topic &amp; is the expected result.

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 40
  • -Receive: 9
  • Posts: 432
    • View Profile
Re: Rexx -> IBM C
« Reply #3 on: October 13, 2018, 08:08:07 pm »
"Boussard&Gavaudan £" -> "Boussard&amp;Gavaudan £". I was expecting a &pound; as a final character now

FWIW: possibly a feature (of a FF) instead of a bug. The real source code uses &pound;, but source code displayed by a FF apparently shows this as £ again. The target browser is Netscape/2 (and anything newer), so I didn't check which character may possible be supported by now without having to HTMLífy this character.

The code now uses strcat() and switch(). No pointers (to pointers) yet, but the global variable isn't that large and I claim to know how that works. Instead of using HTMLName(record.name) I'm using strcpy(buffer,record.name);HTMLName() now.

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 9
  • -Receive: 129
  • Posts: 1866
    • View Profile
Re: Rexx -> IBM C
« Reply #4 on: October 13, 2018, 08:40:26 pm »
It may depend on codepage how FF prints a character. Really everything should be unicode but our port still supports codepages and there may be bugs as Mozilla mostly ripped out codepage support and Bitwise re-added.

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 40
  • -Receive: 9
  • Posts: 432
    • View Profile
Re: Rexx -> IBM C
« Reply #5 on: October 14, 2018, 11:01:27 am »
It may even be a bug of FF 62.0.3, or a feature of DOM. DOM says the source code of the selected text is "£".  The source code of the whole page gets is right by showing the original "&pound;". Perhaps, with ye olde charset=iso-8859-1 or all charsets, is DOM trying to be smart when &pound; may not be required (anymore) to display the GBP character correctly.

The source of the "&" in the same name is always "&amp;".

AFAICT all browsers are showing the HTML page correctly. It could have been my bug because the "£" was a last character of a name, but it wasn't.

If you ever want to copy & paste HTML code, then don't use "DOM source" but use the source code of the whole page. Apparently DOM may show interpreted, improved or simplified source code.

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 40
  • -Receive: 9
  • Posts: 432
    • View Profile
Re: Rexx -> IBM C
« Reply #6 on: October 22, 2018, 03:08:21 pm »
1. With the original batch processing by Rexx, is it "good practice" to NOT use threads unless you can load large file 2 while sorting data of file 1 takes a while? In other words: don't overdo it, if it's not using the message queue?

2. Coding style question. The sample function Thousand(), included below, converts e.g. "12345678" to "12,345,678". Insert the fixed character "," like I've done, or construct a new string here too? Unlike the previous HTML codes, it's a single character which is inserted.


Code: [Select]
#include <os2.h>
#include <stdio.h>
#include <string.h>


char buffer[16];


void Thousand(void)
{
int Count,Corr=0,i,Len;

Len=strlen(buffer);
Count=(Len-1)/3;

buffer[Len+Count]=0;
for (i=Len-1;i>=0;i--)
   {
   Corr++;   
   if (Corr==4)
      {
      Corr=1;
      buffer[i+Count]=(char)',';
      Count--;
      }
   buffer[i+Count]=buffer[i];
   }
return;
}


int main(void)
{
sprintf(buffer,"11111\0");
Thousand();
printf("%s * %s = ",buffer,buffer);

sprintf(buffer,"%u",(ULONG)11111*11111);
Thousand();
printf("%s\n",buffer);
   
return 0;
}


Bogdan

  • Jr. Member
  • **
  • Thank You
  • -Given: 0
  • -Receive: 2
  • Posts: 78
    • View Profile
Re: Rexx -> IBM C
« Reply #7 on: October 22, 2018, 06:23:32 pm »
Dave already mentioned the codepages. But that's not everything.

Even ISO-8859-1 will be interpreted nowadays by most clients as Windows codepage 1252. The posted C snippets are good examples of writing non-portable code, that will cause unpredictable results under different OS/2 configurations. As long the IBM C compiler with the corresponding C library is used it should be possible to be at least portable between several platforms using wide characters. Without checking the locale and codepage only 7-bit ASCII (C locale) can be handled properly.

For OS/2-only operation you need to detect the codepage of the input data (DosQueryCp + UniUconvToUcs or UniStrToUcs), apply some mapping (UCS->GML) and convert to the proper codepage for output.

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 40
  • -Receive: 9
  • Posts: 432
    • View Profile
Re: Rexx -> IBM C
« Reply #8 on: October 23, 2018, 02:27:51 pm »
Dave already mentioned the codepages. But that's not everything.

Even ISO-8859-1 will be interpreted nowadays by most clients as Windows codepage 1252. The posted C snippets are good examples of writing non-portable code, that will cause unpredictable results under different OS/2 configurations. As long the IBM C compiler with the corresponding C library is used it should be possible to be at least portable between several platforms using wide characters. Without checking the locale and codepage only 7-bit ASCII (C locale) can be handled properly.

For OS/2-only operation you need to detect the codepage of the input data (DosQueryCp + UniUconvToUcs or UniStrToUcs), apply some mapping (UCS->GML) and convert to the proper codepage for output.

By definition the software is customized. Input data and output language. The other day a part of a Portugese name ("Indústrias") introduced a first &uacute;, which required a rare update of the code.

Nevertheless the least I can do is to check the codepage indeed, and consider supporting at least the codepages of all involved countries.

Users do have an alternative. "Indústrias" (or a Russian name, or some Japanese name) is a part of a full name, which has an ugly 7-bit ASCII short version. "Bank of China", in more than 7 Chinese bits, could become the 7-bit number 3988". The conversion is quite easy (IF char_of_name > 127 THEN long_name := short_name).

In all cases checking the codepage would be an improvement, because I'm assuming it's always the codepage I'm using right now.

Oops, there's a related bug in at least one of the HTML output files: charset=charset=iso-8859-1

The old iso-8859-1, because Netscape/2 is a target (speed, and how tables look). At the moment all input characters have a &xxx; HTML equivalent, and I've replaced &#nnn; bij &sup1;, &sup2; and &sup3;. Is Netscape/2's UTF-8 or Windows-1250 a better choice for e.g. rather simple French or Spanish names?

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 40
  • -Receive: 9
  • Posts: 432
    • View Profile
Re: Rexx -> IBM C
« Reply #9 on: October 23, 2018, 05:02:48 pm »
Screen output is not worth mentioning. One app displays excluded names.

1. Is DosSetProcessCP() with codepage 850 a way to make sure that my 1/2/3 in superscript are processed correctly?

2. What is the default codepage CONFIG.SYS setting with e.g. a Russian, democratic Chinese or Portugese version of OS/2? 

3. Can you, for testing purposes, change the codepages in CONFIG.SYS without braking anything? I guess you can, but perhaps there's smart OS software which then decides to delete a MANUAL.UK and keeps the MANUAL.RUS.

4. Am I fine with just a check? "if (codepage==850) select (...)"? Codepage 860 doesn't have all required characters, like a superscript 3, so my code and data is quite specific for, in my case, codepage 850. Do keep in mind that I don't want to write a generic TXT2HTML converter. The number of characters to be converted is about 10.

5. The documentation is not that clear. Does default: or good practice require a break;, if there are no more statements?

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 9
  • -Receive: 129
  • Posts: 1866
    • View Profile
Re: Rexx -> IBM C
« Reply #10 on: October 23, 2018, 05:38:50 pm »
3. Can you, for testing purposes, change the codepages in CONFIG.SYS without braking anything? I guess you can, but perhaps there's smart OS software which then decides to delete a MANUAL.UK and keeps the MANUAL.RUS.

One thing that can break is file names on disk, especially with HPFS.

Bogdan

  • Jr. Member
  • **
  • Thank You
  • -Given: 0
  • -Receive: 2
  • Posts: 78
    • View Profile
Re: Rexx -> IBM C
« Reply #11 on: October 23, 2018, 07:18:33 pm »
By definition the software is customized. Input data and output language. The other day a part of a Portugese name ("Indústrias") introduced a first &uacute;, which required a rare update of the code.
Why not simply using a UCS to GML mapping, or whatever is now defined in later HTML DTD.

Quote
Nevertheless the least I can do is to check the codepage indeed, and consider supporting at least the codepages of all involved countries.
It's not possible to map every codepoint directly between different 8-bit codepages.

Quote
Users do have an alternative. "Indústrias" (or a Russian name, or some Japanese name) is a part of a full name, which has an ugly 7-bit ASCII short version. "Bank of China", in more than 7 Chinese bits, could become the 7-bit number 3988". The conversion is quite easy (IF char_of_name > 127 THEN long_name := short_name).
An unsigned char should be used.

Quote
In all cases checking the codepage would be an improvement, because I'm assuming it's always the codepage I'm using right now.

Oops, there's a related bug in at least one of the HTML output files: charset=charset=iso-8859-1

The old iso-8859-1, because Netscape/2 is a target (speed, and how tables look). At the moment all input characters have a &xxx; HTML equivalent, and I've replaced &#nnn; bij &sup1;, &sup2; and &sup3;. Is Netscape/2's UTF-8 or Windows-1250 a better choice for e.g. rather simple French or Spanish names?
For output of ISO-8859-1 the IBM-819 codepage can be used. Windows 1250 is similar to IBM-852 for central/eastern Europe. For IBM-850 or other western codepages it would be better to use 1252.

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 40
  • -Receive: 9
  • Posts: 432
    • View Profile
Re: Rexx -> IBM C
« Reply #12 on: October 26, 2018, 02:16:52 pm »
One thing that can break is file names on disk, especially with HPFS.

So I should be fine, unless I start doing more than strictly required for basis, simple tests/proofs of concept.

1. I cannot check my OS-2 CODEPAGE setting right now, but the default is 850. Does 850,437 mean that I can swap at the OS level (STDOUT, et cetera) to 437 , while an app may use e.g. 860 "internally"?

2. Do all systems have a 850 by default? IOW, would a Portugese OS/2 version have the setting 860,850? Portugese by default, with the international 850 as an option? Or reversed, 850,860.

Andi B.

  • Sr. Member
  • ****
  • Thank You
  • -Given: 1
  • -Receive: 20
  • Posts: 412
    • View Profile
Re: Rexx -> IBM C
« Reply #13 on: October 26, 2018, 02:46:48 pm »
Quote
1. I cannot check my OS-2 CODEPAGE setting right now, but the default is 850. Does 850,437 mean that I can swap at the OS level
help chcp

And check the settings notebook of any program. Here there is a page named 'Sprache'.

Please do not change your codepage setting in your config.sys.

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 40
  • -Receive: 9
  • Posts: 432
    • View Profile
Re: Rexx -> IBM C
« Reply #14 on: October 26, 2018, 03:52:34 pm »
Why not simply using a UCS to GML mapping, or whatever is now defined in later HTML DTD.

Simply? :-\

I'm not sure I need it. The environment, which doesn't change anything I've mentioned, is:

1. Users have to be able to type the 8-bit ASCII name, with any DOS editor (e.g. QBASIC /EDIT) or OS/2 editor.

2. Based on codepage 850, the code may append a superscript 1,2 and/or 3 to names. Those characters aren't available when e.g. codepage 860 is in use.

3. Netscape is the targetted main (local) browser, but the HTML files are available online too. So far there is no specific reason to exclude more desktop environments, like SeaMonkey for OS/2, Qupzilla for Unix or Microsoft's latest Internet Exploder.

4. It looks likes my copy of Netscape for OS/2 stopped counting UTFs at 8 and Windows at 1250. Otherwise I would already have picked UTF-9 to replace ISO-8859-1. Hence the question if UTF-8 or Windows-1250 would be better ... or at least newer than ISO-8859-1.

5. My code does assume that codepage 850 is in use. Checking that is an on-topic coding style improvement: if the codepage is assumed to be 850, then check that the codepage is 850.

6. There may be conflicts, but so far there are none. I'm not using the Portugese codepage 860, but I'm using Portugese names.

7. So far all characters have a &xxxxx; equivalent.

8. So far all characters are supported by codepage 850.

9. The app would have te be customized per user now. Each new high-ACII character in a new name requires a new case: to be appended. If a German user with a German keyboard would type München, which may introduce the first "ü", then I'm willing to insert code to replace their "ü" by an "&uml;". In other words: I'm not supporting any character of my own codepage, and not all characters of of all (single-byte) codepages.

10. Users will use names which are normal to them. If I would visit gazprom.com to find out its official name, I'll visit the EN or DE section, and not the RU section. I'll use Gazprom instead of Gasprom or a Cyrillic name. The reversed would be true too, so for a Russian user I may have to deal with all of ther Cyrillic characters.

It sounds more complicated than it is, but I will check the codepage. If the codepage is not 850, then the question still is if I can always try to stwitch to that codepage to produce my 850-based output, even with some Japanese OS/2 install.

I will look at the "simply"-stuff anyway, if anything to learn about a few APIs, but its added value may be NULL. I don't even support all typable characters of my codepage 850. I guess that the added value will increase if producing &#nnn; codes for all of the world's characters is suprisingly easy and simple.

Netscape remains a requirement, regardless of more modern charsets of other browsers. HTML was the selected output format for "reports", and Netscape for OS/2 is fast for local browsing and has ugly but rather clear TABLEs.

The superscript characters 1,2 and 3, which may be appended to names or text, represent foot notes. So I may have to change that code too, to use e.g. &sup3; instead of an appended single character. That way supporting e.g. codepage 860 will be easier. This change may be quite hard, because the code shouldn't change "&sup1;" to "&amp;sup1;" next.

Quote
It's not possible to map every codepoint directly between different 8-bit codepages.

I've seen that with the superscript characters. Hence the possible rewrite of of the superscript-code... ???

To-do:

1a. Check the codepage
1b. Consider trying to set the codepage to the assumed codepage 850, if it's not 850.

2. Try to change the superscript-code, to avoid those single characters and make it easier to support more codepages.

3a. Take a look at the APIs mentioned earlier.
3b. If it is both easy and useful to use those APIs, then consider doing that it.

Quote
For output of ISO-8859-1 the IBM-819 codepage can be used. Windows 1250 is similar to IBM-852 for central/eastern Europe. For IBM-850 or other western codepages it would be better to use 1252.

As stated earlier, the number of visible Netscape codepage suppport is limited. I'm okay with ISO-8859-1, but of course a more modern alternative supported by Netscape would be even better. Apparently UTF-9 was such an alternative, but Netscape's last offer seems to be UTF-8.

Netscape isn't a must-have, but typically generated HTML files are browsed as local files. With Netscape opening such a file always is a matter of seconds, and output is "designed"with and for Netscape/2.

Oh, yet another reason to write a customized HTML generator for e.g. a theoretical user from Portugal is that generic text will have to be translated too. Column headers, and so on.

The HTML code itself won't be interesting nor on-topic. Rexx isn't that aware of codepages, but can create the same output. Converting the code to C didn't affect the HTML code.

Homework... Check the codepage, consider changing it to 850... Try to svoid single superscript-characters... Study APIs...