Author Topic: JSon to Rexx (stem)  (Read 39707 times)

Fr4nk

  • Guest
Re: JSon to Rexx (stem)
« Reply #15 on: January 08, 2014, 10:25:31 am »
Hiho,

well it is not a real XCenter Widget. I prefer to use vanilla wps-objects. But of course you might drop such an object on your x-center/warpcenter/bubblepad/whatever, get an icon and a fly-over forecast.

I'm not sure about IP localization, maybe we do it as an option - but also manually localization entries (of course). Btw a nice geo-localization service is http://en.utrace.de/api.php - they provide very simple requests and returns XML-based data to get latitude and longitude.

Hopefully I'll find free time for this next weekend.

cu shortly :)
Fr4nk

« Last Edit: January 08, 2014, 03:20:53 pm by Fr4nk »

Jan-Erik Lärka

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 275
  • Karma: +5/-0
    • View Profile
Re: JSon to Rexx (stem)
« Reply #16 on: January 08, 2014, 02:47:19 pm »
Ahh, I followed the link and like it alot! Really nice!

As a matter of fact, my initial post was aimed at you when I wrote it to use the openweather service.  ;)

//Jan-Erik

Below I've added selection 3 and 4 to the xml-parser if someone would need it.
Code: [Select]
parse_xml_tags: PROCEDURE /* xmlString, tag <<, tag_value>, selection> */
    prev_trace = TRACE( 'O' )
    PARSE UPPER ARG xmlString, tag, val, selection
    start_tag = '<'||tag
    INTERPRET "PARSE VALUE xmlString WITH pre'"||start_tag||val||"'extra'>'post"
    SELECT
        WHEN selection = 1 THEN /* tag */
        DO
            IF LENGTH( extra ) > 0 THEN
            DO
                PARSE VALUE SUBSTR( ARG(1), ABS( LENGTH( xmlString ) - LENGTH( post ) + 1 ) ) WITH pre'<'.
                RETURN pre
            END
        END
        WHEN selection = 2 THEN /* tag value */
            RETURN SUBSTR( ARG(1), ABS( LENGTH( xmlString ) - LENGTH( post ) - LENGTH( extra ) ), LENGTH( extra ) - ( RIGHT( extra, 1 ) = '/' ) )
        WHEN selection = 3 THEN /* tag value */
        DO
            tag_val = SUBSTR( ARG(1), ABS( LENGTH( xmlString ) - LENGTH( post ) - LENGTH( extra ) ), LENGTH( extra ) - ( RIGHT( extra, 1 ) = '/' ) )
            INTERPRET "PARSE CASELESS VALUE tag_val WITH .'"||val||'="'||"'ret_val'"||'"'||"'."
            IF LENGTH( ret_val ) > 0 THEN RETURN ret_val
            INTERPRET "PARSE CASELESS VALUE tag_val WITH .'"||val||"='ret_val ."
            IF LENGTH( ret_val ) > 0 THEN RETURN ret_val
            ELSE RETURN tag_val /* What 2 do else?! */
        END
        WHEN selection = 4 THEN /* tag name value */
        DO
            INTERPRET "PARSE VALUE xmlString WITH pre'"||start_tag||" NAME="||'"'||STRIP( val,, '"' )||'"'||" VALUE="||'"'||"'extra'"||'"'||" />'."
            IF LENGTH( STRIP( extra ) ) = 0 THEN
                INTERPRET "PARSE VALUE xmlString WITH pre'"||start_tag||" NAME="||STRIP( val,, '"' )||" VALUE="||"'extra'"||"/'."
            IF LENGTH( STRIP( extra ) ) = 0 THEN RETURN ''
            start_pos = POS( extra, TRANSLATE( xmlString ) )
            IF start_pos > 0 THEN RETURN SUBSTR( ARG(1), start_pos, LENGTH( extra ) )
            ELSE RETURN ''
        END
        WHEN RIGHT( extra, 1 ) <> '/' THEN
        DO
            end_tag = '</'||tag||'>'
            next = LENGTH( xmlString ) - LENGTH( post ) + 1
            open_tag = next
            INTERPRET "PARSE VALUE post WITH pre'"||end_tag||"'post"
            close_tag = LENGTH( xmlString ) - LENGTH( post ) - LENGTH( end_tag ) + 1
            count = COUNTSTR( start_tag||'>', xmlString, open_tag, close_tag ) + COUNTSTR( start_tag||' ', xmlString, open_tag, close_tag )
            DO WHILE count > 0
                next = close_tag + LENGTH( end_tag )
                close_tag = POS( end_tag, xmlString, next )
                IF close_tag = 0 THEN
                    close_tag = LENGTH( xmlString )
                post = SUBSTR( xmlString, next, close_tag - next )
               
                INTERPRET "PARSE VALUE post WITH .'"||start_tag||"'extra'/>'post"
               
                DO WHILE LENGTH( post ) > 0
                    next = close_tag - LENGTH( post )
                    INTERPRET "PARSE VALUE post WITH .'"||start_tag||"'extra'/>'post"
                END
                count = COUNTSTR( start_tag||'>', xmlString, open_tag, close_tag ) + COUNTSTR( start_tag||' ', xmlString, open_tag, close_tag ) - 1
            END
            IF close_tag = 0 THEN close_tag = LENGTH( ARG(1) ) - open_tag
            ELSE close_tag = close_tag - open_tag
            RETURN SUBSTR( ARG(1), open_tag, MAX( 1, close_tag ) )
        END
        OTHERWISE NOP
    END
    CALL TRACE prev_trace
RETURN ''

I've tested this code with regina that is somewhat limited when it come to PARSE, so you can try to replace:
INTERPRET "PARSE VALUE xmlString WITH pre'"||start_tag||" NAME="||'"'||STRIP( val,, '"' )||'"'||" VALUE="||'"'||"'extra'"||'"'||" />'."
with
val = STRIP( val,, '"' )
PARSE  VALUE xmlString WITH pre(start_tag)' NAME="'(val)'" VALUE="'extra'" />'
« Last Edit: January 08, 2014, 03:44:15 pm by Jan-Erik Lärka »

Fr4nk

  • Guest
Re: JSon to Rexx (stem)
« Reply #17 on: January 08, 2014, 03:22:56 pm »
Thank you - I'll check it out.

Jan-Erik Lärka

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 275
  • Karma: +5/-0
    • View Profile
Re: JSon to Rexx (stem)
« Reply #18 on: February 10, 2018, 10:43:07 pm »
New and improved :)
Tested on json from https://glosbe.com/ as well as openweather.

Code: [Select]
/*
 * Filename: fromJSon.cmd
 *   Author: JANERIK
 *  Created: Sat Feb  3 2018
 *  Purpose:
 *  Changes:
 */

IF ARG() > 0 THEN
    IF STREAM( ARG(1), 'C', 'QUERY EXISTS' ) <> '' THEN
        f_name = ARG(1)
    ELSE
        f_name = 'london.json'
ELSE
    f_name = 'london.json'

IF STREAM( f_name, 'C', 'QUERY EXISTS' ) = '' THEN RETURN 1
f_size = STREAM( f_name, 'C', 'QUERY SIZE' )
f_input = CHARIN( f_name, 1, f_size )
CALL STREAM f_name, 'C', 'CLOSE'
!_arrays_! = 'json.'
DO WHILE 0 < RxFromJSon( f_input )
    CALL CHAROUT 'STDERR', 'Parts of the file missing! Press Enter to read it again. '
    PARSE PULL
    f_size = STREAM( f_name, 'C', 'QUERY SIZE' )
    f_input = CHARIN( f_name, 1, f_size )
    CALL STREAM f_name, 'C', 'CLOSE'
END
SAY 'Press <Enter> to show JSON as flat Stem'
PARSE PULL
DO i = 1 TO json.!.0
   SAY json'.!.'i' -> 'json.!.i' => 'VALUE( json.!.i )
END
SAY 'Press <Enter> to show JSON Tree View Stem'
PARSE PULL
CALL RxRecursive 'json'
RETURN 0

RxRecursive: PROCEDURE EXPOSE (!_arrays_!) stem
   stm = ARG(1)
   DO i = 1 TO VALUE( stm'.0' )
      IF SYMBOL( VALUE( stm'.i' ) ) = 'VAR' THEN
         SAY stm'.'i' -> 'VALUE( stm'.i' )' => 'VALUE( VALUE( stm'.i' ) )
      IF DATATYPE( VALUE( stm'.i.0' ), 'W' ) THEN
         CALL RxRecursive stm'.'i
   END
RETURN 0

RxFromJSon: PROCEDURE EXPOSE (!_arrays_!) stem
   IF SYMBOL( 'stem' ) = 'LIT' THEN
      !_stem_! = 'json'
   ELSE
      !_stem_! = STRIP( stem, 'T', '.' )
   IF SYMBOL( '!_arrays_!' ) = 'LIT' THEN
      !_arrays_! = !_stem_!'.'
   !_json_! = !_stem_!
   !_tree_! = !_stem_!
   !_str_! = STRIP( ARG(1) )
   !_delim_! = ',:'
   !_quote_! = '""""'||"''''"
   !_group_! = '[]{}'
   !__tag__! = !_quote_!||!_group_!||!_delim_!
   !_counter_! = 0
   DO FOREVER
      old_trace = TRACE( 'O' )
      !_next_! = 1
      !_stop_! = LENGTH( !_str_! )
      DO !_i_! = 1 TO LENGTH( !__tag__! )
         !_char_! = SUBSTR( !__tag__!, !_i_!, 1 )
         !_pos_!.!_i_! = POS( !_char_!, !_str_! )
         IF 8 < !_i_! THEN
         DO
            DO !_j_! = 1 TO 7 BY 2
               !_k_! = !_j_! + 1
               IF RxIsBetween( !_pos_!.!_j_!, !_pos_!.!_i_!, !_pos_!.!_k_! ) THEN
                  !_pos_!.!_i_! = POS( !_char_!, !_str_!, !_pos_!.!_k_! + 1 )
            END
         END
         ELSE
            !_pos_!.!_i_! = POS( !_char_!, !_str_!, !_next_! )
         
         IF !_i_! < 9 & 0 < !_pos_!.!_i_! THEN
         DO
            IF !_i_! % 2 = 0 THEN
               !_stop_! = MIN( !_stop_!, !_pos_!.!_i_! + 1 )
            !_next_! = !_pos_!.!_i_! + 1
         END
            ELSE !_next_! = 1
         !_sorted_!.!_i_! = !_pos_!.!_i_! !_char_!
      END
      DO !_i_! = 1 TO LENGTH( !__tag__! )
         DO !_j_! = !_i_! + 1 TO LENGTH( !__tag__! )
            IF SUBWORD( !_sorted_!.!_j_!, 1, 1 ) < SUBWORD( !_sorted_!.!_i_!, 1, 1 ) & 0 < SUBWORD( !_sorted_!.!_j_!, 1, 1 ) | 0 = SUBWORD( !_sorted_!.!_i_!, 1, 1 ) THEN
            DO
               !_temp_! = !_sorted_!.!_i_!
               !_sorted_!.!_i_! = !_sorted_!.!_j_!
               !_sorted_!.!_j_! = !_temp_!
            END
         END
         !_ord_!.!_i_! = SUBWORD( !_sorted_!.!_i_!, 1, 1 )
         IF 0 < !_ord_!.!_i_! THEN
            !_chr_!.!_i_! = SUBWORD( !_sorted_!.!_i_!, 2, 1 )
         ELSE
            !_chr_!.!_i_! = ''
      END
      !_i_! = 2
      CALL TRACE old_trace
      IF 0 < POS( !_chr_!.1, ',}]' ) THEN
      DO
         !_lpos_! = LASTPOS( '.', !_json_! ) - 1
         IF 0 < !_lpos_! THEN
            !_json_! = LEFT( !_json_!, !_lpos_! )
         !_lpos_! = LASTPOS( '.', !_tree_! ) - 1
         IF 0 < !_lpos_! THEN
            !_tree_! = LEFT( !_tree_!, !_lpos_! )
      END
      ELSE IF !_chr_!.3 = ':' THEN
      DO
         DROP result
         !_key_! = SUBSTR( !_str_!, !_ord_!.1 + LENGTH( !_chr_!.1 ), !_ord_!.2 + LENGTH( !_chr_!.2 ) - !_ord_!.1 - LENGTH( !_chr_!.1 ) - 1 )
         !_num_! = 3 + ( 0 < POS( !_chr_!.4, !_quote_! ) | SUBSTR( !_group_!, POS( !_chr_!.4, !_group_! ) + 1, 1 ) = !_chr_!.5 )
         !_num_2_! = !_num_! + 1
         !_val_! = STRIP( SUBSTR( !_str_!, !_ord_!.!_num_! + LENGTH( !_chr_!.!_num_! ), !_ord_!.!_num_2_! + LENGTH( !_chr_!.!_num_2_! ) - !_ord_!.!_num_! - LENGTH( !_chr_!.!_num_! ) - 1 ) )
         !_json_! = !_json_!'.'!_key_!
         IF 0 < LENGTH( !_val_! ) | 0 < POS( !_chr_!.4, !_quote_! ) THEN
         DO
            INTERPRET !_json_!' = "'!_val_!'"'
            !_counter_! = !_counter_! + 1
            CALL VALUE !_stem_!'.!.'!_counter_!, !_json_!
            CALL VALUE !_stem_!'.!.0', !_counter_!
            !_val_! = VALUE( !_tree_!'.0' )
            IF \DATATYPE( !_val_!, 'W' ) THEN
               !_val_! = 0
            !_val_! = !_val_! + 1
            CALL VALUE !_tree_!'.0', !_val_!
            !_tree_! = !_tree_!'.'!_val_!
            CALL VALUE !_tree_!, !_json_!
         END
         !_i_! = !_num_! + 1 + ( 0 < POS( !_chr_!.4, !_quote_! ) | SUBSTR( !_group_!, POS( !_chr_!.4, !_group_! ) + 1, 1 ) = !_chr_!.5 )
      END
      ELSE IF 0 < POS( !_chr_!.1, '{[' ) THEN
      DO
         IF !_chr_!.1 = '[' THEN
            !_val_! = 0
         ELSE
            !_val_! = VALUE( !_json_!'.0' )
         IF DATATYPE( !_val_!, 'W' ) THEN
         DO
            !_val_! = !_val_! + 1
            CALL VALUE !_json_!'.0', !_val_!
            !_json_! = !_json_!'.'!_val_!
            CALL VALUE !_tree_!'.0', !_val_!
            !_tree_! = !_tree_!'.'!_val_!
            CALL VALUE !_tree_!, !_json_!
         END
      END
      IF 0 < !_ord_!.!_i_! THEN
         !_str_! = SUBSTR( !_str_!, !_ord_!.!_i_! )
      ELSE LEAVE
   END
RETURN 0 < POS( '.', !_tree_! )
   
RxIsBetween: PROCEDURE
RETURN ARG(1) < ARG(2) & ARG(2) < ARG(3)
« Last Edit: February 11, 2018, 07:14:12 am by Jan-Erik Lärka »

Neil Waldhauer

  • Hero Member
  • *****
  • Posts: 1024
  • Karma: +24/-0
    • View Profile
    • Blonde Guy
Re: JSon to Rexx (stem)
« Reply #19 on: May 20, 2021, 08:32:33 pm »
I'm waking this thread up after a long time. I started to write a REXX program to control the GL.iNet family of routers. These tiny routers can provide wireless support. But the user interface does not work with Firefox 45.

So I use curl and an API to control the router. So far, I can go to some WiFi hotspots, connect to them and surf the internet.

In order to do this, I needed to interpret the JSON file that CURL returns. So here I am. I used this:
  * Filename: JSon2Rx.cmd
 *   Author: Jan-Erik
 *  Created: Sat Feb 23 2013
 *  Purpose: Interpret a JSon-file and place in stem

I found it had a problem with one of the files I got. So I tried this one:
/*
 * Filename: fromJSon.cmd
 *   Author: JANERIK
 *  Created: Sat Feb  3 2018

Amazingly, this hangs my ArcaOS computer. I type

fromJSon curldns.txt

and it never completes. Is there something wrong with my computer, or is fromJSon.cmd doing something wierd?
Expert consulting for ArcaOS, OS/2 and eComStation
http://www.blondeguy.com

Mentore

  • Full Member
  • ***
  • Posts: 152
  • Karma: +4/-0
    • View Profile
Re: JSon to Rexx (stem)
« Reply #20 on: May 21, 2021, 07:55:03 am »
I'm waking this thread up after a long time. I started to write a REXX program to control the GL.iNet family of routers. These tiny routers can provide wireless support. But the user interface does not work with Firefox 45.

So I use curl and an API to control the router. So far, I can go to some WiFi hotspots, connect to them and surf the internet.

In order to do this, I needed to interpret the JSON file that CURL returns. So here I am. I used this:
  * Filename: JSon2Rx.cmd
 *   Author: Jan-Erik
 *  Created: Sat Feb 23 2013
 *  Purpose: Interpret a JSon-file and place in stem

I found it had a problem with one of the files I got. So I tried this one:
/*
 * Filename: fromJSon.cmd
 *   Author: JANERIK
 *  Created: Sat Feb  3 2018

Amazingly, this hangs my ArcaOS computer. I type

fromJSon curldns.txt

and it never completes. Is there something wrong with my computer, or is fromJSon.cmd doing something wierd?

First question: did you try to validate the json file?

A REXX script should not be able to hang OS/2, but I'm afraid this may happen.
Mentore

Lars

  • Hero Member
  • *****
  • Posts: 1271
  • Karma: +65/-0
    • View Profile
Re: JSon to Rexx (stem)
« Reply #21 on: May 21, 2021, 12:39:11 pm »
What does "it never completes" actually mean? Does it even prevent you from doing a Ctrl-C?
The script contains a recursive function. In case of programming error, it could loop forever and also use up all of the stack. But of course, you should still be able to do a Ctrl-C.

Neil Waldhauer

  • Hero Member
  • *****
  • Posts: 1024
  • Karma: +24/-0
    • View Profile
    • Blonde Guy
Re: JSon to Rexx (stem)
« Reply #22 on: May 21, 2021, 08:54:07 pm »
If I catch it in time, I can ctrl-C to stop it. Otherwise only the power switch.

There are C language JSON interpreters, but it was so nice to have the REXX one.

The website at https://jsonformatter.curiousconcept.com/ says the file is valid JSON, and it shows that the file is really not that complex. There are just 11 items, one of which is an array.
Expert consulting for ArcaOS, OS/2 and eComStation
http://www.blondeguy.com

Lars

  • Hero Member
  • *****
  • Posts: 1271
  • Karma: +65/-0
    • View Profile
Re: JSon to Rexx (stem)
« Reply #23 on: May 22, 2021, 11:14:53 am »
Maybe the easiest thing would be to wrap the C code with the REXX Interface. That will give you a good performance and still allow you to use it from REXX. But of course, you'll either need the C Source or a DLL to link to (with a published interface).

Lars

  • Hero Member
  • *****
  • Posts: 1271
  • Karma: +65/-0
    • View Profile
Re: JSon to Rexx (stem)
« Reply #24 on: May 22, 2021, 11:17:06 am »
If I catch it in time, I can ctrl-C to stop it. Otherwise only the power switch.

There are C language JSON interpreters, but it was so nice to have the REXX one.

The website at https://jsonformatter.curiousconcept.com/ says the file is valid JSON, and it shows that the file is really not that complex. There are just 11 items, one of which is an array.
I take it, the recursion never stops and eventually blows the stack. And then there seems to be a bug in the REXX interpreter to hang the whole system.

Jan-Erik Lärka

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 275
  • Karma: +5/-0
    • View Profile
Re: JSon to Rexx (stem)
« Reply #25 on: May 23, 2021, 08:56:24 am »
Hello,

I used:
set rextrace=on
and stepped thorugh the code...
It parse almost all of the file, but it stumble on the last
Code: [Select]
"code":0}and never complete as it can't handle that last section.
The result is that the code skip past and return the same text over and over again in rapid succession, thus eventually look up the computer.

The short term solution would be just to remove what's left, but I'll look into it somewhat to see what I can do about it.

Recursion:
Problems usually result in that the interpreter exit abnormally with a message that contain a call to the same function, though visually offset from the previous. Rexx can not handle that many levels of recursion s0 it's possible to write code that can parse an advanced file. One has to handle such solutions with queues or stacks.

Code: [Select]
call bla( param_1, param_n )
   call bla( param_1, param_n )
      call bla( param_1, param_n )
         call bla( param_1, param_n )
            call bla( param_1, param_n )
               call bla( param_1, param_n )
                  call bla( param_1, param_n )
                     call bla( param_1, param_n )

Jan-Erik Lärka

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 275
  • Karma: +5/-0
    • View Profile
Re: JSon to Rexx (stem)
« Reply #26 on: May 23, 2021, 10:54:19 am »
Ok,

3 issues:

1. The code need that a json file begin with "[{" and end with "}]"
  It can be easily be resolved with

Code: [Select]
   IF LEFT( !_str_!, 1 ) = '{' THEN
      !_str_! = '['||!_str_!||']'
right after
Code: [Select]
!_str_! = STRIP( ARG(1) )
2. The mentioned ending of the file has to be coded into the part that handle ':'
    Tested here and done.

3. The code focus on key'='value (including such lists) and not on value lists only.
    An additional section has to be added that take care of the list
"proxy_serverlist":[...]