• Welcome to OS2World OLD-STATIC-BACKUP Forum.
 

News:

This is an old OS2World backup forum for reference only. IT IS READ ONLY!!!

If you need help with OS/2 - eComStation visit http://www.os2world.com/forum

Main Menu

Rapidshare nagger

Started by ddan, 2008.09.11, 06:06:45

Previous topic - Next topic

ddan

Well, I admit that with the media players now available for OS/2, I've been snagging shows from http://www.tv-mafia.com/, but often forget to keep my one stream from Rapidshare free service busy, so came up with this to nag me (nagging is good if has a point to it):

-------------------------- paste below to party.cmd ------------------------------
/* PARTY helps maximize downloading from Rapidshare by automated nagging. */
/* Tuned to recent changes that eliminate long waits between downloads. */
/* Chirps while 'RapidShare:' is in window list, then monitors .part files. */

call rxfuncadd 'sysloadfuncs', 'rexxutil', 'sysloadfuncs'
call sysloadfuncs

/* monitor this directory for .part files; trill when appear */
mondir= 'C:\TEMP'

/* poll window list every sleeptime secs; chirp while this is present */
monwl= 'RapidShare:'

/* sleep period in seconds; sets how often window list is queried */
sleeptime= 10

/* number of sleeptime loops before checking directory for .part files */
nloops= 3

np= 0
lc= 0
do forever
  fl= 1 /* flag used to wait through the normal countdown period */
  do until notpresent = 1  /* monitor window list for text in monwl */
    call sysqueryswitchlist "wl."
    notpresent= 1
    do n= 1 to wl.0  /* check all entries */
      if pos(monwl, wl.n) > 0 then notpresent= 0
    end
    if notpresent = 0 & fl = 1 then do
      say '"'monwl'" present! Sleeping for 45 seconds before nagging...'
      call syssleep 45
      fl= 0
    end
    if notpresent = 0 then do /* after first loop, periodic chirping */
      call chirp
      call syssleep sleeptime
      lc= 0 /* reset counter so goes right to check / trill */
    end
  end
  if lc = 0 then do
    lc= 0
    rc= sysfiletree(mondir||'\*.part', 'parts.', 'FO')
    say ' 'time('N')'  Previous = 'np'  Now present = 'parts.0
    if np < parts.0 then call trill
    if np > parts.0 then call ended
    np= parts.0
  end
  call syssleep sleeptime
  lc= lc + 1
  if lc > nloops then lc= 0
end

chirp:
do n= 0 to 2; call beep 2500, 50; call syssleep 0.1; end; return

trill:
do n= 0 to 9; call beep 2000 + 200 * n, 50; end; return

ended:  /* once used to this, might have to nag until a key is pressed... */
do n= 9 to 0 by -1; call beep 1000 + 50 * n, 200; end; return

jep

#1
Hello,

ddan, great contribution to the scripts!  :)


some of you readers may try to code something that will download the file automatically while you're at it!

Below you have some parts that may help you write such a script. Feel free to ask if you get stuck or reply if want to make a remark about something.

Please post code that you may have tried here as well so that we can learn from your experiences and mistakes too!  :P

The code segments and instructions may seem difficult to handle, but most of the work has been done and you should be able to find most info to help you out in this forum or if you like in "Rexx Tips & Tricks".
Please be aware that some of the code may be removed or has to be slightly altered to work here.




This code selects a pretended web browser string and tries to download a page that can use session cookies.
temp = Random( 1, 5 )
if temp = 3 then
    cfg.useragent = 'Mozilla/4.0 (compatible; MSIE 7.0b; Windblows NT 6.0)'
else if temp = 2 then
    cfg.useragent = 'Mozilla/5.0 (OS/2; U; Warp 4.5; en-US; rv:1.8b5) Gecko/20051011 Firefox/1.5.0.4'
else if temp = 1 then
    cfg.useragent = 'Mozilla/4.0 (compatible; MSIE 5.0; Windows XPloder) Opera 6.05 [en]'
else if temp = 4 then
    cfg.useragent = 'Internet Exploder 6.01 or something there about ;-)'
else
    cfg.useragent = 'M$ Internet Xploder Harvester v1.0'

if stream( cfg.source_path||cfg.file.to||cfg.file.ext, 'c', 'query exists' ) \= ''  then
    '@del "'||cfg.source_path||cfg.file.to||cfg.file.ext||'" 1>NUL 2>NUL'

'@cls'
say 'Fetching page...' /* just fetch the page/link, no need to post anything here */
'@wget -O '||cfg.source_path||cfg.save_to_file||' -t 10 -c -T 60 -nv --load-cookies '||cfg.cookies||' --keep-session-cookies --save-cookies '||cfg.cookies||' --user-agent="'||cfg.useragent||'" "'||cfg.referer||'"'

after that you'll "just"  ;D interpret the html-text and go to the download page.

assumed that you've read the file into a variable called "infile"

parse value infile with .'<form id="ff" action="'link'" method="post">'.


then you'll write that link to a file (as specified in the variable cfg.post_link) and post it with wget.
'@wget -O '||cfg.source_path||cfg.save_to_file||' -nv -t 10 -c -T 60 --load-cookies '||cfg.cookies||' --keep-session-cookies --save-cookies '||cfg.cookies||' --post-file='||cfg.source_path||cfg.post_link||' --user-agent="'||cfg.useragent||'" --referer='||cfg.referer||' "'||cfg.referer||'"'

delete the posted file and extract the result from the cfg.save_to_file into the same variable as before called "infile" and extract the link and download the actual file.

parse value infile with .'<form id="dlf" action="'link'" method="post">'.



Explanation to some of the variables used.
cfg.cookies point to a file called "cookies.txt" that should be filled with information about cookies and/or use existing cookies, e.g. "C:\PROGRAMS\firefox\cookies.txt" use cookies retrieve already with you web browser (firefox) while you also can start fresh with a new one in the current folder.

cfg.source_path let it point to the same directory as the script with
parse value . . prog
cfg.source_path = filespec( 'D', prog )||filespec( 'P', prog )


cfg.referer At first you'll set it to the page you want to visit, but then it should often point to the page you're at, as some sites look at that info, and if you omit it, may give you some dummy page in return instead of the one you asked for.

cfg.save_to_file define the file name where the reply should be saved into to make it easier to code.

cfg.post_link where you save the text containing the link that has to be posted to the webserver that hold the page.

Note:
You may see that the code to parse the links look very similar, so try to create a function that you can use for both versions. Hint: I'd use Interpret, but how?  ???

//Jan-Erik

ddan

Hmm. Here's how Rapidshare works at present ── they've quit requiring a capcha
code, and omit an 80 minute or so waits between downloads, which is what makes
the nagger useful, but MIGHT resume those at any time, so I hesitate to put
more effort in ── Load the entry page, click on "Free User"; second page has a
45 second or so wait after it comes up (using Javascript), requires a second
click on that, and a third on the download dialog, because for some reason,
possibly by intent, the "automatic" save is grayed out.

I don't have a clue as how to simulate the clicks, nor whether the Javascript
delay could be simulated. Indeed, I assume that those are present precisely to
foil robots. (HEY! I fail the Turing test! I'm a robot and didn't know it!)

But a large increase of automation is possible by copy/paste links into a text
file, then having a program take those out in coordination with the nagger
logic, and launch Firefox with a nagger beep. The program will delete entries
from the text file when corresponding files appear in the directory (having a
non-zero size that indicates done).

I've found in programming and general computer use that even a partial
solution which works is better than wishing for a total solution, even if
that's to come, because a) actual problems are rarely as you conceive, and b)
the act of solving a problem even partially illuminates better solutions.

Catchphrase: baby steps, baby steps.

jep

#3
Hello,

The nagging script really serves the purpose, one can go on with the next download faster than before. Thanks!
It's a neat script with that show how to use beep. Beep can be used for alot of purposes. One can use it to indicate when something goes wrong, but also to get the users attention, as in this script, when a task take a long time and one doesn't know when it may be done in advance.

Regarding the part with timeout and clicking links:
Link + cookie and sometimes a special param or two is what you need to provide to simulate a click on such pages, If you use "livehttpheaders" then you'll see what to look for. All has to be in the page you look at or otherwise the web browser wouldn't be able to fetch the next page.
If they're really clever they'll place a variable wait period to make it even more difficult.  ;D
But as the timeout is written in javascript, it's something that can be done in rexx too
call SysSleep( found_and_parsed_timeout )

But as you stated, it can take some time to get it. On the other hand, someone else may want it too, and the one that write it learn rexx and how web pages are built (in this case).

//Jan-Erik

ddan

As I feared, Rapidshare has changed yet again and returned to delays between allowed downloads. Modified program complete below, too complex to specify edits. Still useful to prompt, though a bit confused when other d/ls are going too. You should hit a key to end the nagging before opening the Rapidshare window; this is intentional for purposes of nagging, but I already dislike it, may modify to watch automatically.

----------------------------------------------------------------------------------
/* PARTY helps maximize downloading from Rapidshare by automated nagging. */
/* Chirps while 'RapidShare:' is in window list, then monitors .part files. */
/* October 2008: changes, 15 minute waits between downloads, NAGS for new. */
/* now less useful with more than rapidshare files being d/led (by FireFox) */

call rxfuncadd 'sysloadfuncs', 'rexxutil', 'sysloadfuncs'
call sysloadfuncs

/* monitor this directory for .part files; trill when appear */
mondir= 'C:\TEMP'
/* sleep period in seconds; sets how often window list is queried */
sleepqwl= 10
/* poll window list for this string every sleepqwl secs */
monwl= 'RapidShare:'
/* seconds to wait after monwl is present before chirping */
after= 75
/* number of sleepqwl loops sets period of checking dir for .part files */
nloops= 3
/* wait minutes between allowed downloading (for small files, may be less) */
between= 15
/* period of NAGGING loop after between file wait, make "ended" sound */
NAGtime= 60

np= 0
lc= 0
do forever
  fl= 1 /* flag used to wait through the normal countdown period */
  do until notpresent = 1  /* monitor window list for text in monwl */
    call sysqueryswitchlist "wl."
    notpresent= 1
    do n= 1 to wl.0  /* check all entries */
      if pos(monwl, wl.n) > 0 then notpresent= 0
    end
    if notpresent = 0 & fl = 1 then do
      say '"'monwl'" present! Sleeping for 'after' seconds before nagging...'
      call syssleep after
      fl= 0
    end
    if notpresent = 0 then do /* after first loop, periodic chirping */
      call chirp
      call syssleep sleepqwl
      lc= 0 /* reset counter so goes right to check / trill */
    end
  end
  if lc = 0 then do
    lc= 0
    rc= sysfiletree(mondir||'\*.part', 'parts.', 'FO')
    say ' 'time('N')'  Previous = 'np'  Now present = 'parts.0
    if np < parts.0 then call trill
    if np > parts.0 then do
      say 'Sleeping for 'between' minutes (assumes only RS files downloading)'
      do n= between to 1 by -1  /* however, since url is placed in EA, it's */
        call charout, n'... '   /* possible to ignore non-RS files; the fix */
        call syssleep 60        /* awaits it becoming a problem for me */
      end
      say
      say 'You MUST hit a key to end the NAG loop!'
      do while chars() = 0  /* _REALLY_ NAG LOOP */
        call ended
        do nag= 1 to 10
          call syssleep NAGtime / 10 /* for faster response */
          call charout, 'Nag '
          if chars() > 0 then do
            say 'Nagging ended; monitoring for 'monwl
            nag= 11
          end
        end
      end
      k= sysgetkey('noecho') /* read just to clear chars() test */
    end
    np= parts.0
  end
  call syssleep sleepqwl
  lc= lc + 1
  if lc > nloops then lc= 0
end

chirp:
do n= 0 to 2; call beep 2500, 50; call syssleep 0.1; end; return

trill:
do n= 0 to 9; call beep 2000 + 200 * n, 50; end; return

ended:  /* once used to this, might have to nag until a key is pressed... */
do n= 9 to 0 by -1; call beep 1000 + 50 * n, 200; end; return