Page 1 of 1

Re: ZEVO site content

PostPosted: Fri Aug 15, 2014 5:47 pm
by ilovezfs
I went ahead and downloaded the whole ZEVO forum, including all of the attachments. Here's the zip file for anyone who wants it:

zevo.getgreenbytes.com.zip
ZEVO forum archive
(30.03 MiB) Downloaded 751 times

Though the ZEVO forum is shutdown, for future reference if anyone is trying to download an entire phpBB site, including attachments only accessible to logged-in users, here's what I did.

  1. Log in to the site using this script:

    https://gist.github.com/ilovezfs/b7e283d4fcdd6f6bcf24

    Code: Select all
    #!/bin/sh
    # phpbb-auto-login.sh copyleft Fabio Z 2009
    # login & logout user to phpBB forum boards
    # http://www.zoros.org/wiki/index.php?title=A_script_to_login_phpbb_forum_using_wget
     
     
    if [ $# -le 1 ] || [ $# -gt 3 ]; then
     echo "Usage $0 URL user pass --> to login"
     echo "Usage $0 URL user      --> to logout"
    fi
     
    PHPBB_URL=http://$1
    USER=$2
    PASS=$3
     
     
    #--------------------------- login ---------------------------
    if [ $# -eq 3 ] ; then
     
     wget --save-cookies=./session-cookies-$USER $PHPBB_URL/ucp.php?mode=login -O - 1> /dev/null 2> /dev/null
     
     SID=`cat ./session-cookies-$USER | grep _sid | cut -d$'\011' -f7`
     
     echo "Login $USER --> $PHPBB_URL SID=$SID"
     
     wget --save-cookies=./session-cookies-$USER \
     --post-data="username=$USER&password=$PASS&redirect=index.php&sid=$SID&login=Login" \
     $PHPBB_URL/ucp.php?mode=login --referer="$PHPBB_URL/ucp.php?mode=login" \
     -O - 1> /dev/null 2> /dev/null
     
    fi
     
     
    #-------------------------- logout --------------------------
    if [ $# -eq 2 ] ; then
     
     
     SID=`cat ./session-cookies-$USER | grep _sid | cut -d$'\011' -f7`
     
     echo "Logout $USER <-- $PHPBB_URL SID=$SID"
     
     wget --load-cookies=./session-cookies-$USER "$PHPBB_URL/ucp.php?mode=logout&sid=$SID" --referer="$PHPBB_URL/index.php" -O -  1> /dev/null 2> /dev/null
     
    fi


    Code: Select all
    ./phpbb-auto-login.sh zevo.getgreenbytes.com/forum ilovezfs mypassword

    That will output a file named "session-cookies-ilovezfs."

  2. Download the entire site:

    Code: Select all
    wget \
        --mirror \
        --output-file=log.txt \
        --execute robots=off \
        --reject 'faq.php*,mcp.php*,memberlist.php*,posting.php*,report.php*,search.php*,ucp.php*,viewonline.php*,viewtopic.php*p=*,*bookmark=*,*watch=*,*view=print*,*start=0*,*sid*,*start=0*' \
        --no-parent \
        --page-requisites \
        --adjust-extension \
        --convert-links \
        --referer='http://zevo.getgreenbytes.com/forum/index.php' \
        --load-cookies session-cookies-ilovezfs \
        --save-cookies session-cookies-ilovezfs \
        --keep-session-cookies \
        'http://zevo.getgreenbytes.com/forum'

    Here is the same command, all on one line:

    Code: Select all
    wget --mirror --output-file=log.txt --execute robots=off --reject 'faq.php*,mcp.php*,memberlist.php*,posting.php*,report.php*,search.php*,ucp.php*,viewonline.php*,viewtopic.php*p=*,*bookmark=*,*watch=*,*view=print*,*start=0*,*sid*,*start=0*' --no-parent --page-requisites --adjust-extension --convert-links --referer='http://zevo.getgreenbytes.com/forum/index.php' --load-cookies session-cookies-ilovezfs --save-cookies session-cookies-ilovezfs --keep-session-cookies 'http://zevo.getgreenbytes.com/forum'

  3. If you are feeling a need to be particularly tidy, you can log out as well:

    Code: Select all
    ./phpbb-auto-login.sh zevo.getgreenbytes.com/forum ilovezfs

@gperrin, note that "--html-extension" is deprecated, and has been replaced by "--adjust-extension." Also, "--no-clobber" is incompatible with "--convert-links," and you will get the warning "Both --no-clobber and --convert-links were specified, only --convert-links will be used."