• Welcome to OS2World OLD-STATIC-BACKUP Forum.
 

News:

This is an old OS2World backup forum for reference only. IT IS READ ONLY!!!

If you need help with OS/2 - eComStation visit http://www.os2world.com/forum

Main Menu

Google translation

Started by jep, 2008.06.09, 15:02:33

Previous topic - Next topic

jep

Marked as: Advanced
Hello,

here's some code for you to try out... it can help you translate between the languages available.
You may want to rewrite some of the reconition code to allow alternative ways of specifying the right language. You have to type each language right to make it work, with upper case of the first letter etc. as it look now.

Save the code to googletranslate.cmd and try it.

Usage: Googletranslate English German This is my first attemt to use Google for translations

Note that it may not be 100% accurate, but you should be able to get a fairly decent translation anyway that you just have to modify a bit.

Prerequisite: Unicode conversion library as specified in similar topic.

Self study: Ways to extract data from html code.

//Jan-Erik

/* Google translation */
'@echo off'

/* Load Rexx Utility functions */
CALL RxFuncAdd 'SysLoadFuncs', 'REXXUTIL', 'SysLoadFuncs'
CALL SysLoadFuncs

/* Load UniCode and CodePage functions */
/* Alex Taylors addon, http://hobbes.nmsu.edu/pub/os2/dev/rexx/rxuls_052.zip */
CALL RxFuncAdd 'ULSLoadFuncs', 'RXULS', 'ULSLoadFuncs'
CALL ULSLoadFuncs

lang.from = SUBWORD( ARG(1), 1, 1 )
lang.to = SUBWORD( ARG(1), 2, 1 )
cfg.from_word = SUBWORD( ARG(1), 3 )
CurrCP = ULSQueryLocaleItem( 'iCodePage' )
GlotCP = 1004

if stream( 'in.tmp', 'c', 'close' ) = 'READY:' then
  retval = SysFileDelete( 'in.tmp' )
if stream( 'in.tmp', 'c', 'query exists' ) \= ''  then
  retval = SysFileDelete( 'in.tmp' )

lang.0 = 24
lang.1.code = 'auto'
lang.1 = 'Detect Automatically'
lang.2.code = 'ar'
lang.2 = 'Arabic'
lang.3.code = 'bg'
lang.3 = 'Bulgarian'
lang.4.code = 'zh-CN'
lang.4 = 'Chinese'
lang.5.code = 'hr'
lang.5 = 'Croatian'
lang.6.code = 'cs'
lang.6 = 'Czech'
lang.7.code = 'da'
lang.7 = 'Danish'
lang.8.code = 'nl'
lang.8 = 'Dutch'
lang.9.code = 'en'
lang.9 = 'English'
lang.10.code = 'fi'
lang.10 = 'Finnish'
lang.11.code = 'fr'
lang.11 = 'French'
lang.12.code = 'de'
lang.12 = 'German'
lang.13.code = 'el'
lang.13 = 'Greek'
lang.14.code = 'hi'
lang.14 = 'Hindi'
lang.15.code = 'it'
lang.15 = 'Italian'
lang.16.code = 'ja'
lang.16 = 'Japanese'
lang.17.code = 'ko'
lang.17 = 'Korean'
lang.18.code = 'no'
lang.18 = 'Norwegian'
lang.19.code = 'pl'
lang.19 = 'Polish'
lang.20.code = 'pt'
lang.20 = 'Portuguese'
lang.21.code = 'ro'
lang.21 = 'Romanian'
lang.22.code = 'ru'
lang.22 = 'Russian'
lang.23.code = 'es'
lang.23 = 'Spanish'
lang.24.code = 'sv'
lang.24 = 'Swedish'


do i = 1 to lang.0
   if lang.from = lang.i then /* modify to allow similar ways of decribe the language */
   do
     cfg.from_lang = lang.i.code
     leave i
   end
end
do i = 1 to lang.0
   if lang.to = lang.i then /* modify to allow similar ways of decribe the language */
   do
     cfg.to_lang = lang.i.code
     leave i
   end
end
call lineout 'out.tmp', "hl="||cfg.to_lang||"&ie=UTF8&text="||translate( ULSConvertCodepage( cfg.from_word, CurrCP, GlotCP ), '+', ' ' )||"&sl="||cfg.from_lang||"&tl="||cfg.to_lang
if stream( 'out.tmp', 'c', 'close' ) = 'READY:' then
  '@wget -O "in.tmp" -U "Google Harvester" --post-file=out.tmp -nv -t 2 -c -T 60 http://translate.google.com/translate_t'
if stream( 'out.tmp', 'c', 'query exists' ) \= ''  then
  retval = SysFileDelete( 'out.tmp' )
tgt_size = stream( 'in.tmp', 'c', 'query size' )
retval = charin( 'in.tmp', 1, tgt_size )
parse value retval WITH .'dir="ltr" id=source>'sze'</textarea>'.'<div id=result_box dir="ltr">'tgt'</div>'.
call stream 'in.tmp', 'c', 'close'
Say ToNLS( sze, GlotCP, CurrCP )
Say 'has been translated to:'
Say ToNLS( tgt, GlotCP, CurrCP )
Return 0

ToNLS: procedure
retval = ARG(1)
do while pos( '&#', retval ) > 0
  parse value retval with pre'&#'val';'post
  retval = pre||d2c(val)||post
end
Return ULSConvertCodepage( retval, ARG(2), ARG(3) )

ddan

Hello jep,

First, must say that I always enjoy your REXX code, even when not immediately used. I think REXX is the way to put program building blocks together as we were promised back in the early 90's. As I'm experimenting with elsewhere, it's just amazing to be able to control complex programs and the WPS from a little bit of text.

This looks interesting, but I'm just not seeing where "in.tmp" comes from. Does this already
exist, you have it hard-coded for testing?

RobertM

#2
Hi ddan,

If I am ready the code correctly, jep checks for an old copy of in.tmp, if there, deletes it:

if stream( 'in.tmp', 'c', 'close' ) = 'READY:' then
  retval = SysFileDelete( 'in.tmp' )
if stream( 'in.tmp', 'c', 'query exists' ) \= ''  then
  retval = SysFileDelete( 'in.tmp' )

then creates a new copy with:

'@wget -O "in.tmp" -U "Google Harvester" --post-file=out.tmp -nv -t ......

...to use in the rest of the script.

Rob


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


jep

#3
Hello,

quite right, I reused the code from the interglot translation and used filenames (in.tmp and out.tmp) that resemble variables. Not that clever, but convenient.

WGet has to post a file to retrieve the translation, so the code prepare that file, but before it can, it has to ensure that there's no garbage left from the previous run.

The bonus from reusing that code was that it included the logics to remove the temporary file from some previous download to start clean on the next run. It look odd and backwards, but if you spot that it's not a variable but a filename and that it's for removal then it's easier to see why it's there. The approach was selected to be able to analyze the downloaded file in case of a failure in how I built the code and the logic within. It may be better to remove the file after a successful run, but that is a design decision that may have (dis)advantages too, especially if the code hasn't been tested that well as in this case.  ;D
It took about 5 to 10 tests to get it right as I knew what to look for with http://liveHTTPHeaders.mozdev.org.

It can be quite handy to examine the code in a text editor with syntax highlighting. It can give a visual hint with colors and style how different subtle sections may behave. ePM, PMFTE or similar does a great job, the disadvantage is that one can tend to stretch the boundaries on how to write rexx code, some code just does magic, perhaps to much at times.  :P

The code in short:

  • Load required support functions (rexxutil and unicode support)
  • Store info about input and output language to use
  • Text after that should be translated
  • Get current codepage on system
  • Delete remains from a previous run if there
  • Decide what input and output language to use
  • Write the info to a temporary file to post
  • Post the file to the translation service
  • Get the size of the file in return
  • Read the whole file to memory
  • Extract the parts that conaitn the info we're interested in ( sensetive to changes in the future )
  • Convert special characters and the text to the current codepage and display the result

http://translate.google.com/ require that one state the user agent or it will refuse to return any information... so I called it "Google Harvester", but almost anything seem to work. Wonder if they monitor what user agents use their service? :-)

Someone interested in rewriting the code to use cURL?! cURL is library (rexx dll) and equivalence to wget that behave better e.g. in visual rexx tools.

//Jan-Erik

ddan

Believe I've got it now: the text to be translated is given on the command line from the 3rd word, not in either temporary file. Couldn't understand why you were deleting BOTH files without any obvious use. Needed to look up exactly that, anyway.

Thanks. I may even get round to trying this.