ZEVO site content

Issues specific to non-OpenZFS pools (i.e., ZEVO & MacZFS).

Re: ZEVO site content

Postby ilovezfs » Fri Aug 15, 2014 5:47 pm

I went ahead and downloaded the whole ZEVO forum, including all of the attachments. Here's the zip file for anyone who wants it:

zevo.getgreenbytes.com.zip
ZEVO forum archive
(30.03 MiB) Downloaded 752 times

Though the ZEVO forum is shutdown, for future reference if anyone is trying to download an entire phpBB site, including attachments only accessible to logged-in users, here's what I did.

  1. Log in to the site using this script:

    https://gist.github.com/ilovezfs/b7e283d4fcdd6f6bcf24

    Code: Select all
    #!/bin/sh
    # phpbb-auto-login.sh copyleft Fabio Z 2009
    # login & logout user to phpBB forum boards
    # http://www.zoros.org/wiki/index.php?title=A_script_to_login_phpbb_forum_using_wget
     
     
    if [ $# -le 1 ] || [ $# -gt 3 ]; then
     echo "Usage $0 URL user pass --> to login"
     echo "Usage $0 URL user      --> to logout"
    fi
     
    PHPBB_URL=http://$1
    USER=$2
    PASS=$3
     
     
    #--------------------------- login ---------------------------
    if [ $# -eq 3 ] ; then
     
     wget --save-cookies=./session-cookies-$USER $PHPBB_URL/ucp.php?mode=login -O - 1> /dev/null 2> /dev/null
     
     SID=`cat ./session-cookies-$USER | grep _sid | cut -d$'\011' -f7`
     
     echo "Login $USER --> $PHPBB_URL SID=$SID"
     
     wget --save-cookies=./session-cookies-$USER \
     --post-data="username=$USER&password=$PASS&redirect=index.php&sid=$SID&login=Login" \
     $PHPBB_URL/ucp.php?mode=login --referer="$PHPBB_URL/ucp.php?mode=login" \
     -O - 1> /dev/null 2> /dev/null
     
    fi
     
     
    #-------------------------- logout --------------------------
    if [ $# -eq 2 ] ; then
     
     
     SID=`cat ./session-cookies-$USER | grep _sid | cut -d$'\011' -f7`
     
     echo "Logout $USER <-- $PHPBB_URL SID=$SID"
     
     wget --load-cookies=./session-cookies-$USER "$PHPBB_URL/ucp.php?mode=logout&sid=$SID" --referer="$PHPBB_URL/index.php" -O -  1> /dev/null 2> /dev/null
     
    fi


    Code: Select all
    ./phpbb-auto-login.sh zevo.getgreenbytes.com/forum ilovezfs mypassword

    That will output a file named "session-cookies-ilovezfs."

  2. Download the entire site:

    Code: Select all
    wget \
        --mirror \
        --output-file=log.txt \
        --execute robots=off \
        --reject 'faq.php*,mcp.php*,memberlist.php*,posting.php*,report.php*,search.php*,ucp.php*,viewonline.php*,viewtopic.php*p=*,*bookmark=*,*watch=*,*view=print*,*start=0*,*sid*,*start=0*' \
        --no-parent \
        --page-requisites \
        --adjust-extension \
        --convert-links \
        --referer='http://zevo.getgreenbytes.com/forum/index.php' \
        --load-cookies session-cookies-ilovezfs \
        --save-cookies session-cookies-ilovezfs \
        --keep-session-cookies \
        'http://zevo.getgreenbytes.com/forum'

    Here is the same command, all on one line:

    Code: Select all
    wget --mirror --output-file=log.txt --execute robots=off --reject 'faq.php*,mcp.php*,memberlist.php*,posting.php*,report.php*,search.php*,ucp.php*,viewonline.php*,viewtopic.php*p=*,*bookmark=*,*watch=*,*view=print*,*start=0*,*sid*,*start=0*' --no-parent --page-requisites --adjust-extension --convert-links --referer='http://zevo.getgreenbytes.com/forum/index.php' --load-cookies session-cookies-ilovezfs --save-cookies session-cookies-ilovezfs --keep-session-cookies 'http://zevo.getgreenbytes.com/forum'

  3. If you are feeling a need to be particularly tidy, you can log out as well:

    Code: Select all
    ./phpbb-auto-login.sh zevo.getgreenbytes.com/forum ilovezfs

@gperrin, note that "--html-extension" is deprecated, and has been replaced by "--adjust-extension." Also, "--no-clobber" is incompatible with "--convert-links," and you will get the warning "Both --no-clobber and --convert-links were specified, only --convert-links will be used."
ilovezfs
 
Posts: 232
Joined: Thu Mar 06, 2014 7:58 am

Return to Non-OpenZFS Pools

Who is online

Users browsing this forum: No registered users and 5 guests

cron