OS2World OLD-STATIC-BACKUP Forum

OS/2 - SIGs => Rexx => Topic started by: jep on 2008.04.14, 14:27:41

Title: Online translation, UniCode conversion etc.
Post by: jep on 2008.04.14, 14:27:41
Marked as: Advanced
Hello,

See example on how to fetch pages, parse the text to retrieve translations between the 6 languages available on InterGlot ( http://www.interglot.com (http://www.interglot.com) )

You may use this example to try other translation services as well.

Note: InterGlot return text as CodePage 1252 (1004) while OS/2 and eComStation often use 850, 437 etc. The included script depend on Alex Taylors addon (see link in code below) to convert text between codepages so that you can get the correct translation. I've also tested other similar addons available and they work as well, you just have to change some params and function names to use those instead.

//Jan-Erik

Prerequisite: Save code to iGlot.cmd
Usage: iGlot English Deutsch redundant

/* Interglot translation */
'@echo off'

/* Load Rexx Utility functions */
CALL RxFuncAdd 'SysLoadFuncs', 'REXXUTIL', 'SysLoadFuncs'
CALL SysLoadFuncs

/* Load UniCode and CodePage functions */
/* Alex Taylors addon, http://hobbes.nmsu.edu/pub/os2/dev/rexx/rxuls_052.zip */
CALL RxFuncAdd 'ULSLoadFuncs', 'RXULS', 'ULSLoadFuncs'
CALL ULSLoadFuncs

lang.from = SUBWORD( ARG(1), 1, 1 )
lang.to = SUBWORD( ARG(1), 2, 1 )
cfg.from_word = ULSTransform( SUBWORD( ARG(1), 3 ), 'lower' )
GlotCP = 1004

if stream( 'in.tmp', 'c', 'close' ) = 'READY:' then
  retval = SysFileDelete( 'in.tmp' )
if stream( 'in.tmp', 'c', 'query exists' ) \= ''  then
  retval = SysFileDelete( 'in.tmp' )

lang.0 = 6
lang.1 = 'Nederlands' /* Dutch */
lang.2 = 'English' /* English */
lang.3 = 'Deutsch' /* German */
lang.4 = 'Français' /* French */
lang.5 = 'Español' /* Spanish */
lang.6 = 'Svenska' /* Swedish */

do i = 1 to lang.0
   if translate( lang.from ) = translate( lang.i ) then
   do
     cfg.from_lang = i
     leave i
   end
end
do i = 1 to lang.0
   if translate( lang.to ) = translate( lang.i ) then
   do
     cfg.to_lang = i
     leave i
   end
end

call lineout 'out.tmp', "word="||ULSConvertCodepage( cfg.from_word,, GlotCP )||"&SrcLang="||cfg.from_lang||"&DstLang="||cfg.to_lang
if stream( 'out.tmp', 'c', 'close' ) = 'READY:' then
  '@wget -O "in.tmp" --post-file=out.tmp -nv -t 2 -c -T 60 http://interglot.com/resultpage.php'
if stream( 'out.tmp', 'c', 'query exists' ) \= ''  then
  retval = SysFileDelete( 'out.tmp' )
a = 1
i = 1
j = 0
do while lines( 'in.tmp' ) > 0
  retval = linein( 'in.tmp' )
  parse value retval WITH .'scodes["'num'"] = "'Scodes.a'"'.
  if length( Scodes.a ) > 0 then
  do
    Scodes.num = Scodes.a
    a = a + 1
  end
  parse value retval WITH .'SrcStar['num']  = "'SrcStar.i'";'.
  if length( SrcStar.i ) > 0 then
  do
    SrcStar.i = ULSConvertCodepage( SrcStar.i, GlotCP )
    parse value linein( 'in.tmp' ) with .'SrcCodes['.'"'SrcCodes.i'";'.
    if pos( 'v', SrcCodes.i ) > 0 then
      SrcCodes.i.conj = 1
    else
      SrcCodes.i.conj = 0
    i = i + 1
  end
  parse value retval WITH .'DestWords['chk']['num']="'Dest.Wrd'";'.
  if length( Dest.Wrd ) > 0 then
  do
    parse value linein( 'in.tmp' ) WITH .'DestCodes['chk_2']['num_2']="'Dest.Cde'";'.
    if chk = chk_2 & num = num_2 then
    do
      if SrcStar.j.0 > num then
        j = j + 1
      num = num + 1
      SrcStar.j.0 = num
      SrcStar.j.num = ULSConvertCodepage( Dest.Wrd, GlotCP )
      SrcCodes.j.num = Dest.Cde
      if pos( 'v', SrcCodes.j.num ) > 0 then
        SrcCodes.j.num.conj = 1
      else
        SrcCodes.j.num.conj = 0
    end
  end
end
call stream 'in.tmp', 'c', 'close'
drop Dest.
Scodes.0 = a
SrcStar.0 = i - 1


do i = 1 to SrcStar.0
  num = left( SrcCodes.i, 1 )
  if DataType( num, 'N' ) then
    if Scodes.num \= 'SCODES.'num then
      say cfg.from_lang SrcCodes.i '['Scodes.num'] ('SrcStar.i')' SrcCodes.i.conj
    else
      say cfg.from_lang SrcCodes.i '('SrcStar.i')' SrcCodes.i.conj
  else
    say cfg.from_lang SrcCodes.i '('SrcStar.i')' SrcCodes.i.conj
  do j = 1 to SrcStar.i.0
    Say cfg.to_lang SrcCodes.i.j SrcStar.i.j SrcCodes.i.j.conj
  end
end
Title: Re: Online translation, UniCode conversion etc.
Post by: jep on 2008.04.17, 20:55:17
Marked as: Normal
Alternative + additions

Hello,

Here's my original code to do the translation of words between different languages. Save the attached file and download the following rexx .dll for UNICode support from http://hobbes.nmsu.edu/pub/os2/dev/rexx/rexxuni20.zip (http://hobbes.nmsu.edu/pub/os2/dev/rexx/rexxuni20.zip) or modify the code to use the package referred to in the previous post.

Test the code by entering:
InterGlot börja
and you'll see how it'll not only retrieve the translation to the word but also the conjugates, thesaurus so you should soon find out what "börja" mean in your language. ;)

Note: The example contain a call to the function Translation with 2 numbers, 112 (Swedish) and 25 (English), change those to translate between other languages. See possible combinations in the last function of InterGlot.cmd

//Jan-Erik
Title: Re: Online translation, UniCode conversion etc.
Post by: kim on 2008.04.17, 21:27:26
Really nice example Jan-Erik; BIG thanks for sharing with us mortals that can use your example for getting started with rexx.
Title: Re: Online translation, UniCode conversion etc.
Post by: jep on 2008.05.21, 17:32:28
Hello again.

Would be most welcome if you try to write code for http://www.thefreedictionary.com/ (http://www.thefreedictionary.com/) yourselves.

Most of the code posted in this topic should be possible to reuse, but the interpretation part has to be rewritten to support the different structure that page have.

Howto:
Test a random word that come to mind, e.g. "separate"
Download a page with you webbrowser and save it as a text file to you disk
Open the file with e, epm, fte or some other text editor
Determine what words/parts you want to extract and what tags surround them, e.g.
<h1>the_word_I_want_to_extract</h1>
Create a new rexx script file
Copy code ( if you like ) that read the whole file to memory at once
Write a function that parse the file and extract what you want and structure it up in related variables.
parse input with .'<h1>'variable.4.word'</h1>'input
Post your findings, even if they're only partially correct or if you stumble upon some problematic part so other people can see, discuss and suggest improvments to.

//Jan-Erik