Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?

Re: pack and unpack multiple arrays with one common repeat prefix

by Eily (Monsignor)
on Jun 15, 2017 at 15:36 UTC ( #1192868=note: print w/replies, xml ) Need Help??

in reply to pack and unpack multiple arrays with one common repeat prefix

I think your last proposition (first unpacking the count, then generating a template using it) is probably your best option, most readable.

You might be able to achieve what you want with something like the code below, but it really lacks clarity when you want to read something longer than a byte:

use strict; use warnings; use Data::Dump qw( pp ); my $str = pack "C (a)*", 4, 'a'..'z'; pp $str; pp unpack 'C/a @0 CXC/x/a @0 CXCXC/x/x/a', $str;
"\4abcdefghijklmnopqrstuvwxyz" ("abcd", "efgh", "ijkl")

  • You already know what C/a does. => the stack is ("abcd") and pos = 5
  • @0 goes back to the start. => the stack is ("abcd") and pos = 0
  • CXC reads the first value, goes back one byte, and reads it again => the current list is ("abcd", 4, 4) and pos = 1
  • /x removes the last value in the list (4) and uses it as a repeat count for x, which just skips a byte (so /x ignores four bytes) => the list is ("abcd", 4) and pos = 5
  • /a removes the last value in the list (4 again), and uses it as a repeat count for a => the list is ("abcd", "efgh") and pos = 9
  • @0 goes back the start, CXCXC reads the count three times, /x/x uses the count twice to skip bytes, and /a reads the last substring.

This gets harder when you want to skip more than one byte at a time. This can be done with x[V] which means "skip as many bytes as there are in V". But you can't write C/x[V] because the x would have two counts (one explicit, and one from the stack). So you have to write C/(x[V]) where you apply C times the group "skip as many bytes as in V".
However, at least in my version of perl, you can't use: CXC /(x[a]) /a because skipping a group seems to change the stack and add an element that is used instead of the correct value. It kind of looks like after C/a CXC /(x[a]) the stack is actually ("abcd", 4, ()) instead of ("abcd", 4). That extra element is removed if you add "xX" which skips one byte forward, and skips one byte backward. This doesn't do anything useful, except the next /a does what you want.

This leads to : pp unpack 'C/a @0 CXC /(x[a]) xX /a @0 CXC /((x[a])2) xX /a', $str;
Once again, your last proposition is probably a better solution.

Replies are listed 'Best First'.
Re^2: pack and unpack multiple arrays with one common repeat prefix
by hexcoder (Curate) on Jun 15, 2017 at 16:35 UTC
    Thanks for the very detailed explanation!

    I never understood the '@'- and 'x'- examples from the very tight 'packed' pack() description before.

    I agree, that my last template is more readable, but my table driven decoder is just too dumb to do it in two steps...

      Speaking of the solution in two steps, it can be written like this:

      $count = unpack "C", $str; @values = unpack "x C$count (V$count)2"

      my table driven decoder is just too dumb to do it in two steps...
      I have no idea what you mean by "table driven decoder", but maybe we can help you make it work in two steps instead? Or is there some code that you can't modify that can just be fed a pack template and nothing else?

      I suppose you can't change the way the data is packed either? Because if you pack only V values, you just have: my (@values) = unpack "C/(V3)", $str;

        I wrote a decoder that reads binary telegrams (or messages) with different types (meaning different structures) from a log file. For each telegram type the table has an unpack template that defines the structure of the telegram and a list of names for its fields. The structure is unpacked and the values are formatted with their names to make a readable string representation describing the content of the telegrams.

        Currently this works well with simple types and single arrays prefixed with their length. The structure of the telegrams are given and part of an interface. I agree, there could be simpler unified packings, but that is what I have to deal with.

        Maybe it is a good idea to allow 'two step unpacking', in order to enable the decoding of more complex structures. Thanks, I have to think about it.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1192868]
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others examining the Monastery: (6)
As of 2023-12-05 17:42 GMT
Find Nodes?
    Voting Booth?
    What's your preferred 'use VERSION' for new CPAN modules in 2023?

    Results (27 votes). Check out past polls.