Expect is very powerful, but often difficult to get right (at least for me).  One thing that helps to debug is to print a lot of output.  I also find myself referring to the man page for Perl/Expect:  man Expect, which is a partial answer to your first question.

One trick I've recently started using is to send a unique pattern to the remote host, expect that pattern back, and then just parse all of the text surrounding it.

Here's an example:

use strict;
use warnings;


#############
## Globals ##
#############
my $sendex_index = 0;


#################
## Subroutines ##
#################

#
#  sendex
#
#   Inputs:  $1 - the Expect object
#            $2 - (optional) a verbosity level, if greater than 1,
#                 displays verbose results in the send_expect() subrou
+tine.
#            $3 - (optional) a timeout in seconds
#            $4 - (optional) a command to send to the remote host
#
#  Results:  Sends the command (if any) to the remote host, and then
#            tries to send a unique string of the format "Okay<number>
+.",
#            where <number> will start with '1', and increase on each 
+call
#            to this subroutine.
#
#            Returned will be the same info hash as returned by the
#            call send_expect().
#
sub sendex {
    my ($exp, $vflag, $timeout, $cmd) = @_;
    $timeout ||= 3;
    ++$sendex_index;
    ($cmd || "") and $exp->send("$cmd\n");

    my $okay = "echo -n Okay; echo -n $sendex_index; echo '.'";
    my $pat  = "Okay${sendex_index}.";

    $vflag ||= 0;
    return send_expect($exp, $timeout, $okay, $pat, $vflag);
}


#
#  send_expect
#
#   Inputs:  $1 - the Expect object
#            $2 - a timeout in seconds
#            $3 - a command to send to the remote host
#            $4 - the expected pattern to match
#            $5 - (optional) a verbosity level, if greater than 1,
#                 displays verbose results
#
#  Results:  Sends the command (if any), and attempts to send a unique
#            string of the format "Okay<number>.", where <number> will
#            start with '1', and increase on each call.
#
#            Returned will be the same info hash as returned by the
#            call send_expect().
#
sub send_expect {
    my ($exp, $timeout, $cmd, $pat, $vflag) = @_;

    $vflag ||= 0;
    if ($vflag > 1) {
        print "=" x 79, "\n";
        print "Sending text .... '$cmd'\n";
    }

    $exp->send("$cmd\n");

    if ($vflag > 1) {
        print "Expected text ... '$pat'\n";
    }

    my $origstr = $pat;
    my $b_regex = ($pat =~ s,^/(.+)/$,$1,)? 1: 0;
    my @expargs = $b_regex? (-re, $pat): ($pat);

    my $result = $exp->expect($timeout, @expargs);
    $result or die "sent '$cmd', did not get '$origstr'\n";

    my $before = $exp->before;
    my $match  = $exp->match;
    my $after  = $exp->after;

    my $c_strip = sub {
        my ($s_str) = @_;
        $$s_str =~ s/\e\]0;//g;
        $$s_str =~ s/\e\[[^m]*m//g;
    };

    $c_strip->(\$before);
    $c_strip->(\$match);
    $c_strip->(\$after);

    if ($vflag > 1) {
        print "Full response:\n";
        print "${before}${match}${after}\n";
        print "=" x 79, "\n\n";
    }

    return { 'before' => $before, 'match' => $match, 'after' => $after
+ };
}
[download]

Each time you call sendex(), it echos a different pattern on the remote host (although it does so by echoing pieces of the pattern, so that the call to expect() doesn't succeed too early!):

Okay1
Okay2
Okay3
...
[download]

and as soon as it receives the expected pattern, it sends back a hash containing the 'before', 'match', and 'after' patterns, which you can then parse to suit your needs.