Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Odometer pattern iterator (in C). (Updated.)

by BrowserUk (Patriarch)
on May 29, 2015 at 06:58 UTC ( [id://1128230]=perlquestion: print w/replies, xml ) Need Help??

BrowserUk has asked for the wisdom of the Perl Monks concerning the following question:

At its basic, I want to generate all the numbers from 0 .. N; that have M bits set. Ie. N=5, M=3, generate:

00111 01011 01101 01110 10011 10101 10110 11001 11010 11100

But not necessarily in that particular order. Ie. order is immaterial so long as they are all generated.

What I don't want to do is iterate 0 .. N and then count the bits and then eliminate.

More to the point, what I really need is the positions of the set bits, not the numbers containing them.

Update: What I actually want is the indices of the set bits; Ie. the second group of numbers in each line below:

11100 [0 1 2] 11010 [0 1 3] 11001 [0 1 4] 10110 [0 2 3] 10101 [0 2 4] 10011 [0 3 4] 01110 [1 2 3] 01101 [1 2 4] 01011 [1 3 4] 00111 [2 3 4]

Final complication is that this will be called from Inline::C, for speed, which means calling back into Perl for each iteration would be expensive. In other words, it'll need to be coded in C.

As C doesn't like returning multiple values; and cannot construct functions-on-the-fly the way we often do with Perl; the interface I'm thinking of is:

typedef struct { int N, M; char *posns. } POSNS; POSNS *initGenerator( int N, int M ) { POSNS *p = malloc( sizeof( POSNS ) ); p->N = N; p->M = M; p->posns = malloc( M ); for( i = 0; i < M; ++i ) p->posns[ i ] = N - M + i; return posns; } char *iterGenerator( POSNS *p ) { if( done) { free( p->posns ); free( p ); return NULL; } // modify p->posns; ... return p->posns; } POSNS p = initGenerator( n, m ); while( c = iterGenerator( p ) ) { // use c[]. }

I think this is going to be an odometer pattern iterator; possibly recursive; but beyond that my mind is blank.

I can convert a Perl solution to C, but it would need to avoid things that can't be done in C, (like generating subs on the fly).

Thoughts? Suggestions? Condemnations :)


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked

Replies are listed 'Best First'.
Re: Odometer pattern iterator (in C).
by Anonymous Monk on May 29, 2015 at 09:00 UTC
    #!/usr/bin/perl # http://perlmonks.org/?node_id=1128230 use strict; $_ = '00111'; 1 while print("$_\n"), s/.*\K01(.*)/10 . reverse $1/e;

    gives

    00111 01011 01101 01110 10011 10101 10110 11001 11010 11100

      Wow! This is impressively simple. I would have never thought that this was possible.

        yes, we have a very good AnonymousMonk's contributions nowadays!..

        There are no rules, there are no thumbs..
        Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.
      Quite clever!
        Indeed, quite clever.

      As cute as that is (and it is damn cute!), calling back into Perl and then invoking the regex engine won't make for a performant solution.

      Also, as I've highlighted in my update above, I want the positions of the set bits, not the bit patterns themselves.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
      In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked

        Translating that regex into C is trivial: (1) Find rightmost occurence of substring '01' (we are done if there is none), (2) change '01' to '10', (3) reverse the string to the right of that.

        Finding the indices of the '1's is also simple.

        A chance to brush off my C :)

        // inc_c - http://perlmonks.org/?node_id=1128230 #include <stdlib.h> #include <stdio.h> #define M 5 // place #define N 3 // number of elements wanted static int place[N]; int step(void) { for(int i = 0; i < N; i++) { if(i < N - 1 ? place[i] < place[i + 1] - 1 : place[i] < M - 1 ) { place[i]++; for(int j = i - 1; j >= 0; j--) place[j] -= place[0]; return 1; } } return 0; } int main(int argc, char **argv) { int i; int more = 1; for(i = 0; i < N; i++) place[i] = i; while(more) { for(i = 0; i < N; i++) printf(" %d", place[i]); putchar('\n'); more = step(); } exit(0); }
Re: Odometer pattern iterator (in C).
by hdb (Monsignor) on May 29, 2015 at 07:05 UTC

    Exactly what I recently posted. The array @logical is what you are looking for. Should be simple to translate into C.

    Update: This generates all numbers up to 2**N-1 that have M bits set. Is that what you want? Your spec is unclear.

      Thanks to your having identified what I was after as "combinations of n things taken k at a time"; which just hadn't dawned on me, I went looking around in the guts of Algorithm::Combinatorics and translated his XS version of __next_combinations() to C, and arrived at:

      #include <stdio.h> #include <stdlib.h> #include <malloc.h> #include "..\C\mytypes.h" int nextCombination( U8 *tuple, int len_tuple, int max_n ) { int i, j; I32 offset = max_n - len_tuple; for( i = len_tuple; i >= 0; --i ) { I32 n = tuple[ i ]; if( n < i + offset ) { tuple[ i ] = ++n; for( j = i + 1; j <= len_tuple; ++j ) tuple[ j ] = ++n; return i; } } return -1; } void main( int argc, char **argv ) { U32 n = argc > 1 ? atoi( argv[ 1 ] ) : 5; U32 m = argc > 2 ? atoi( argv[ 2 ] ) : n>>1; U32 i, t = 1; U8 *tuple = malloc( m ); for( i = 0; i < m; ++i ) tuple[ i ] = i; for( i = 0; i < m; ++i ) printf( "%u ", tuple[ i ] ); printf( "\n" + ); while( nextCombination( tuple, m-1, n-1 ) >= 0 ) { for( i = 0; i < m; ++i ) printf( "%u ", tuple[ i ] ); printf( +"\n" ); ++t; } printf( "Iters:%I64u\n", t ); free( tuple ); return; }

      Which works very nicely and very quickly.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
      In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked

      I've updated the above to try and clarify.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
      In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
Re: Odometer pattern iterator (in C).
by Discipulus (Canon) on May 29, 2015 at 07:13 UTC
    The only think i can say, if you are speaking about odometer, is that the example in High Order perl does contains only plain Perl (if you want to translate it to C).
    sub increment_odometer { my @odometer = @_; my $wheel = $#odometer; # start at rightmost wheel until ($odometer[$wheel] < 9 || $wheel < 0) { $odometer[$wheel] = 0; $wheel--; # next wheel to the left } if ($wheel < 0) { return; # fell off the left end; no more sequences } else { $odometer[$wheel]++; # this wheel now turns one notch return @odometer; } }


    .. just in case you missed this.
    L*
    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.

      Unfortunately, that iterate combinations_with_repeats() rather than just combinations() (See Algorithm::Combinatorics for the difference.)


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
      In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
Re: Odometer pattern iterator (in C).
by salva (Canon) on May 29, 2015 at 07:52 UTC
    M=5
    4 3 2 1 0 5 3 2 1 0 5 4 2 1 0 5 4 3 1 0 5 4 3 2 0 5 4 3 2 1 6 3 2 1 0 6 4 2 1 0 6 4 3 1 0 6 4 3 2 0 6 4 3 2 1 6 5 2 1 0 6 5 3 1 0 6 5 3 2 0 6 5 3 2 1 6 5 4 1 0 6 5 4 2 0 6 5 4 2 1 6 5 4 3 0 6 5 4 3 1 6 5 4 3 2 7 3 2 1 0 ...

      I'm sure this is telling me something important; I just have no clue what :)


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
      In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
Re: Odometer pattern iterator (in C).
by johngg (Canon) on May 29, 2015 at 08:59 UTC

    This node may do what you want.

    Cheers,

    JohnGG

      As I said at the time. It's nice for pure perl, by avoiding the generate to eliminate steps; but it a) generates a sub on the fly -- impossible in C; b) calls into the regex engine, which isn't cheap.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
      In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
Re: Odometer pattern iterator (in C). (Updated.)
by Laurent_R (Canon) on May 31, 2015 at 21:21 UTC
    Hi BrowserUk,

    I know you won't like me saying this, but I think that this comment:

    It appears to me that the actual output (in the revised OP ...), is “give me all 3-tuples in the domain 0..5 such that none of the values in the tuple are duplicated.” You can construct the bit-sequences from this if you need them.
    from your very dear friend sundialsvc4 is actually interesting and relevant and the idea of using a recursive function also makes some sense (although, to tell the truth, I fail to see the relevance of sundialsvc4's subsequent comments on the eight-queens problem).

    So, forgetting the bits in your original question and directly searching for all unique N-tuples in a set of M unique elements, I came up with this try:

    use strict; use warnings; my $M = 5; my @a = 1..$M; my $N = 3; get_all_tuples ($N, [], @a); sub get_all_tuples { my ($level, $valref, @a) = @_; if ($level <= 1) { print "@$valref $_\n" for @a; } else { while (@a) { my $i = shift @a; push @$valref, $i; get_all_tuples ($level-1, $valref, @a); pop @$valref; } } }
    which produces the following output:
    $ perl N_among_M.pl 1 2 3 1 2 4 1 2 5 1 3 4 1 3 5 1 4 5 2 3 4 2 3 5 2 4 5 3 4 5
    Changing M to 10 and N to 4:
    $ perl N_among_M.pl 1 2 3 4 1 2 3 5 1 2 3 6 1 2 3 7 1 2 3 8 1 2 3 9 1 2 3 10 1 2 4 5 ... 5 7 9 10 5 8 9 10 6 7 8 9 6 7 8 10 6 7 9 10 6 8 9 10 7 8 9 10
    Removing the printing of the solutions and just storing them in an array, and printing only the array count at the end, and using M = 20 and N = 10, the program completes in just slightly over half a second:
    $ time perl N_among_M.pl 167960 real 0m0.524s user 0m0.499s sys 0m0.015s
    Computing almost 1.7m solutions in just over half a second is not too bad, it seems to me, but I have no idea on the dimensions of the actual problem you are trying to solve. This solution can easily be ported to C, I think, if performance needs to be higher.

    Sorry to come back relatively late on the subject, but this weekend of mine was primarily aimed at filling my tax returns. One of the most boring task each year, I am happy that I could enjoy doing some Perl programming afterwards, this is really more pleasing and more fun.

      I know you won't like me saying this,

      I have no problem with you saying it; though I somewhat fear what the consequence might be given previous examples.

      this comment: “give me all 3-tuples in the domain 0..5 such that none of the values in the tuple are duplicated.” ... is actually interesting

      So, the question is, where did he steal it from?

      and the idea of using a recursive function also makes some sense (although, to tell the truth, I fail to see the relevance of ... the eight-queens problem

      Exactly. None.

      directly searching for all unique N-tuples in a set of M unique elements, I came up with this

      You came up with a solution.

      His posts are like those "Your Stars" astrology pieces in newspapers; so vague that you can read just about anything into them.

      But your solution bears no resemblance to the "pseudo-code" he posted; and that bears no resemblance to any language I know; nor anything vaguely workable.

      Computing almost 1.7m solutions in just over half a second is not too bad, it seems to me, but I have no idea on the dimensions of the actual problem you are trying to solve. This solution can easily be ported to C, I think, if performance needs to be higher.

      Three thoughts:

      • My current solution (posted and refined by Anonymonk above) is iterative; which generally outstrip recursive solutions.

        (That's why compilers attempt to convert recursion to iteration.)

      • How would you convert that recursive solution into an iterator, as requested in the title and described in the OP?

        It's relatively easy to do in Perl; you just write it to take a callback (code block), but much harder to do in C.

      • How does it do on 32/16?

        Does it find all 601 million solutions? Does it come close to doing so in 5s?


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
      In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
        How does it do on 32/16?

        Does it find all 601 million solutions? Does it come close to doing so in 5s?

        Yes, it finds 601 million solutions, but very very far from 5 sec.
        $ time perl N_among_M.pl 601080390 real 35m14.046s user 35m11.691s sys 0m0.108s

      I think that count is wrong. I compute 184756 for 20/10.

        Yes, right, I made a mistake when removing the printing out of the solutions and just counting the solutions. The result is now:
        $ time perl N_among_M.pl 184756 real 0m0.560s user 0m0.545s sys 0m0.015s

      For 20/10 with my C version - http://perlmonks.org/?node_id=1128364

      $ time ./inc_c count 184756 real 0m0.006s user 0m0.003s sys 0m0.000s
Re: Odometer pattern iterator (in C).
by Laurent_R (Canon) on May 29, 2015 at 11:18 UTC
    Perhaps this:
    $ perl -E 'say for grep {@b = split //; $c = 0; for $i (@b){$c++ if +$i}; $c == 3} glob "{0,1}{0,1}{0,1}{0,1}{0,1}";' 00111 01011 01101 01110 10011 10101 10110 11001 11010 11100

    Update: sorry, forget it, I had missed or forgotten that you don't want to iterate 0 .. N and then count the bits and then eliminate. Saw it just after having posted.

Re: Odometer pattern iterator (in C). (Updated.)
by Anonymous Monk on May 29, 2015 at 22:28 UTC

    Perhaps this might help?

      The problem is unpacking the bits to get the indices is relatively expensive compared to generating the indices directly.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
      In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
A reply falls below the community's threshold of quality. You may see it by logging in.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1128230]
Approved by hdb
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others surveying the Monastery: (4)
As of 2024-03-29 12:56 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found